Testing Strategy

A test exists to fail when behavior breaks — and to stay green when only structure changes. Everything here serves that: test behavior through public interfaces, at the cheapest level that would catch the bug, with real collaborators where affordable and deterministic doubles where not. Agents fail at testing in predictable ways — asserting on mocks, snapshotting everything, weakening or deleting failing tests, sleeping instead of waiting — and this skill exists to stop those.

Before you start

If .agents/project-context.md exists (or .claude/project-context.md), read it first and treat it as ground truth for stack, platforms, scale, design system, and constraints. Only ask for what it does not cover. If it does not exist and the task is larger than a quick fix, suggest running the project-context skill — but never block on it.

Core principles

1. Follow the codebase's existing test conventions first. Same runner, folder layout, naming, fixtures, and helpers; a second test framework is a cost everyone pays.
2. Every test names the break it catches. If you can't say which realistic bug makes it fail, it's coverage theater.
3. Test behavior through public interfaces. Tests coupled to private structure break on every refactor and miss real bugs.
4. Cheapest level that catches the bug. Unit for logic, integration for wiring and queries, end-to-end for a few critical journeys.
5. Real over fake over mock. Use the real thing when fast and deterministic, a fake at I/O boundaries, and a mock only when the interaction is the contract.
6. Deterministic or it doesn't ship. Control time, randomness, IDs, network, and ordering; a flaky test trains everyone to ignore red.
7. Never weaken, skip, or delete a failing test to get green without stating why the test — not the code — was wrong.

Workflow

• Survey. Find the test runner, config, existing test folders, fixtures/factories, and CI job. Run the suite once to learn baseline time and failures.
• Classify the system (table below) to pick the test shape; note what already exists at each level.
• List behaviors and risks for the change: happy path, edge cases (empty, max, unicode, time zones, concurrency), error paths, authorization.
• Assign each behavior a level using "What to test where". Prefer extending existing test files.
• Write the test first where practical (bug fixes always: reproduce, watch it fail, then fix).
• Prove it can fail. Break the code (flip a condition, return early) and confirm the test goes red; restore.
• Check determinism. Run new tests repeatedly (e.g., 10–20 times, and in random order if the runner supports it). Any failure → fix before merging.
• Report using the Output format.

Pick the shape by system type

What to test where

Writing good tests

• Arrange–Act–Assert, one behavior per test, named as a sentence: rejects_refund_after_30_days, "shows empty state when there are no invoices".
• Expected values are literals you derived by hand, not computed by the code under test or its helpers.
• No logic in tests — no loops or conditionals deciding what to assert; use table-driven cases instead.
• Test data via factories/builders with sensible defaults, overriding only fields the test cares about.
• Assert on outcomes users or callers see: return values, persisted state, emitted events, rendered text by accessible role — not private fields or call order of internals.
• Each test sets up and cleans its own state; no dependence on test order or data left by another test.

Test doubles

Rules:

• Don't mock what you don't own. Wrap the third-party SDK in your adapter; fake the adapter in unit tests; test the adapter itself against the vendor's sandbox or a recorded/intercepted HTTP layer.
• Don't mock the database for query code. Run the real engine (Testcontainers, an ephemeral database, or the framework's transactional test DB). SQL mocks pass while production queries fail.
• Intercept HTTP at the network boundary (MSW, WireMock, nock, respx, OkHttp MockWebServer) rather than mocking the HTTP client library's methods.
• Mocks mirror real shapes completely — partial mock objects hide the fields production reads.
• If mock setup is longer than the test, switch to an integration test with real components.

More: references/test-doubles.md.

Deterministic tests

Flaky test policy

1. A flaky test is a bug — in the test or in the product (race conditions are real).
2. Reproduce by running it in a loop (e.g., 50–100 times) and under load/parallelism; capture seed, trace, logs.
3. Fix the root cause (see references/flaky-tests.md for the cause → fix table).
4. If it can't be fixed today, quarantine it — excluded from the blocking run, still executed and reported — with an owner, a ticket, and a deadline. Never quarantine silently.
5. CI retries only with reporting. Automatic retry may unblock a merge, but a pass-on-retry is recorded as flaky and tracked; retries must never make flakiness invisible.

TDD and other techniques — when

Details: references/advanced-techniques.md.

End-to-end, visual, and mobile (defaults)

• Playwright is the default web e2e tool unless the repo already uses another. Locate by role/label/text (getByRole('button', { name: 'Pay' })), use web-first assertions that auto-wait, never waitForTimeout. Seed data through APIs, reuse authenticated state, capture traces on retry. Keep the suite to critical journeys.
• Visual regression: fixed viewport, fonts loaded, animations disabled, time frozen, dynamic regions masked; review diffs like code.
• Mobile: unit-test view models and domain logic on the JVM/host; UI tests with Compose testing/Espresso and XCUITest for a few flows; screenshot tests for key screens; cross-platform flows (Maestro, Detox for React Native) where the project uses them. Details: references/e2e-visual-mobile.md.

Coverage

• Coverage is a signal, not a goal. Low coverage on critical code is a finding; high coverage proves nothing about assertions.
• Gate on diff coverage (coverage of new/changed lines, at a threshold the team agrees on) rather than a global percentage that invites filler tests.
• For critical modules, use mutation score to check that tests actually detect changes.
• Ask the review question: "Will this test fail when the code is broken?"

Gotchas

• Asserting on the mock. expect(mockSend).toHaveBeenCalled() when the real outcome (record saved, message rendered) is observable. Assert outcomes; if you're checking a mock exists, delete the assertion.
• Tests that re-implement the code. Computing the expected value with the same formula or helper guarantees a pass. Use literals.
• Snapshotting everything. Huge component snapshots get updated with -u without reading. Snapshot small, stable outputs only.
• Weakening assertions or adding .skip to get green. Forbidden without an explicit reason that the test was wrong; report it in the summary.
• Catching the error in the test so a throwing path "passes". Use the runner's expect(...).rejects / pytest.raises / assertThrows and assert the error type.
• sleep(2000) to wait for async work. Slow and still flaky. Await the condition.
• Mocking the database or ORM in repository tests. Use a real engine in a container.
• Real network calls in unit/integration tests (payment sandboxes, LLM APIs). Slow, flaky, and costly; intercept.
• Tests that pass on first run and were never seen failing. Break the code once to prove the test works.
• One giant e2e test per feature. Push edge cases down to unit/integration; e2e covers the journey.
• Testing framework behavior (that the router calls your handler, that useState updates). Test your contract.
• Testing LLM output with exact string equality. Use schema checks and eval suites (ai-native-architecture).
• Adding a new test framework or assertion library when the repo already has one.

Output format

For a test plan:

System type → shape: <e.g., web app → trophy>
Behaviors and risks:
- <behavior> → <level> → <double strategy> → <break it catches>
Determinism controls: <clock/seed/network/data isolation>
Not tested (and why): <explicit gaps>

After writing tests:

Added/changed tests: <file: test names>
Proved failing: <how you broke the code and saw red>
Runs: <command, pass count over N repeated runs>
Removed/skipped/weakened tests: <none, or each with the reason>
Not checked: <what remains unverified>

References

Related skills

• clean-code — refactoring safely once characterization tests are in place.
• code-review — reviewing tests in a PR for strength and test gaming.
• reliability — testing failure modes, timeouts, and resilience in production-like conditions.
• ai-native-architecture — eval suites for LLM features.
• mobile-architecture — test architecture for mobile apps.