Go test effectiveness

Design tests that detect plausible defects and remain stable when irrelevant implementation details change. Treat coverage as reachability information, not proof of test quality.

A useful test has four parts:

1. behavior worth protecting;
2. a realistic defect that could violate it;
3. an independent reason for the expected result;
4. an assertion that fails when the defect is present.

Express each important test as:

This test protects [behavior] and fails if [defect] occurs.

If neither blank can be filled with a concrete answer, reconsider the test.

Workflow

1. Select the task mode

Classify the request before acting:

• Author or fix: write or change tests, then validate them.
• Review or audit: assess existing tests and report findings. Do not edit files or run destructive or expensive checks unless the user asks.
• Advice: recommend test strategies or explain tradeoffs without requiring repository access.

Honor the requested scope. Do not turn a review or advice request into an implementation task.

2. Inspect the relevant evidence

For repository-backed work, start with the affected implementation, its nearest existing tests, and the available contract or requirement. Expand to public APIs, documentation, call sites, and bug reports when risk or uncertainty requires more evidence. Check go.mod and repository guidance before selecting libraries or commands. For advice without repository access, use the context supplied by the user and state material assumptions.

State the externally meaningful behavior. Do not start from uncovered lines.

3. Identify plausible failure modes

Name defects that fit the code and domain, such as:

• an off-by-one boundary or wrong comparison;
• missing validation or a state change after failed validation;
• use of the wrong account, user, tenant, or resource;
• incorrect transaction semantics;
• ordering dependence, stale state, overflow, or dropped transitions;
• malformed parser input or serialization incompatibility;
• a race, deadlock, goroutine leak, or cancellation bug.

Use historical production bugs when available to identify realistic failure modes. They are stronger fault evidence than invented edge cases, but they establish the expected behavior only when the incident record or accepted fix documents it.

4. Establish the oracle

For every important assertion, identify why the expected result is correct. Prefer sources in this order:

1. requirement or acceptance criterion;
2. protocol or external specification;
3. domain invariant;
4. incident record or accepted fix that documents the expected behavior;
5. independently written reference model or calculation;
6. consumer or provider contract;
7. human-reviewed golden fixture;
8. previously trusted implementation.

Reject circular oracles. Do not derive the expected value by calling the code under test, copying its algorithm, mirroring its conditionals, or accepting snapshots generated by the same implementation without review.

If no independent source establishes the expected behavior, ask the user to clarify the contract before authoring a normative assertion. For a behavior-preserving refactor, an explicitly scoped characterization test may record current observable behavior without claiming that behavior is correct. State that limitation, and do not turn uncertain behavior into a permanent contract without user agreement. During review or advice work, report the missing oracle instead of inventing one.

5. Choose tests from the failure mode

Do not default to table-driven unit tests. Use the smallest combination that exposes the identified risks.

Read only the guidance needed for the identified risks:

• Read property-testing.md when designing or reviewing properties, generators, shrinking, or stateful models.
• Read test-strategies.md when choosing or changing the test type, boundary, or supporting tool.
• Read effectiveness-validation.md when auditing a suite or using coverage, mutation testing, negative controls, historical bugs, or CI checks.

6. Implement, review, or recommend

For author or fix work, write the smallest strong test set. Prefer a few independent assertions over many near-duplicate cases. Assert observable behavior unless an interaction is itself the contract.

For review or audit work, evaluate the existing tests against the identified behaviors, defects, and oracles. Report findings before proposing changes, and do not edit unless asked.

For advice, recommend the smallest combination of test types that addresses the stated risks. Separate known facts from assumptions.

Follow repository conventions first. Do not add a dependency merely because a reference mentions it. For new property tests, prefer Rapid when the repository has no existing standard. For HTTP boundaries, use net/http/httptest when appropriate. Use real disposable dependencies when a mock would hide the behavior under test.

Keep expected values visibly independent from production logic. Construct valid generated values directly rather than generating garbage and skipping most cases.

7. Demonstrate sensitivity

A passing test only proves compatibility with the current implementation. Where practical, show that the test fails when its protected behavior is broken.

Use one of:

• mutation testing;
• a temporary isolated negative control;
• the historical buggy revision;
• a deliberately faulty reference implementation.

Run deliberate faults only in an isolated worktree, copied package, mutation tool, or historical revision. Never alter a file that contains unrelated uncommitted changes. If safe isolation is unavailable, skip the negative control and report that limitation. Afterward, verify that no intentional defect or temporary change remains in the user's working tree.

For a regression fix, the strongest evidence is:

buggy behavior: test fails
fixed behavior: test passes

When a mutation survives, determine the missing behavioral guarantee. Do not add a case merely to kill the mutant syntax.

8. Check stability at the relevant scope

For author or fix work, run focused tests first, then repository-standard checks. For review or audit work, run checks only when requested or needed to substantiate a finding. Typical commands include:

go test ./path/to/pkg
go test -shuffle=on -count=10 ./path/to/pkg
go test -race ./path/to/pkg
go test -cover ./...

Use shuffle and repetition for new tests with possible order dependence. Use the race detector for exercised concurrent paths. Prefer deterministic synchronization over arbitrary sleeps. Investigate flakes instead of masking them with retries.

9. Report the evidence

Match the report to the task mode.

For author or fix work, name:

• tests added or changed and behaviors protected;
• oracle sources;
• sensitivity evidence, if performed;
• commands run and results;
• significant gaps or checks not run.

For review or audit work, lead with findings ordered by severity and include file references. Name the behavior, plausible defect, oracle weakness, or validation gap behind each finding. Separate requested changes from optional suggestions.

For advice, give the recommendation, why it fits the risk, required oracle, and material assumptions or limitations.

Review gate

Before finishing, answer the questions relevant to the selected task mode:

1. What behavior does each meaningful test protect?
2. What plausible defect would make it fail?
3. Where did each important expected result come from?
4. Are the assertions stronger than mere execution or absence of panic?
5. Does the chosen test type match the failure mode?
6. Would irrelevant refactoring leave the test intact?
7. Was sensitivity demonstrated where practical?
8. Are integration, concurrency, property, or fuzz concerns covered where relevant?
9. If checks were run, did the tests pass repeatedly at the appropriate scope?

For author or fix work, improve the test when a relevant answer is unknown or ask the user when the contract is unclear. For review or advice work, report the gap. Do not call the work complete solely because go test passes or coverage increased.