write-tests

Writing tests is easy. Writing the ones worth having is not. The reason behind when the test goes first: <this-skill-directory>/references/why.md.

The criterion: test what hurts if it fails

You don't test for coverage, you test for consequence. Before writing a test, ask what happens if that breaks in production and nobody notices.

One test per real bug. It is the rule with the best return there is: the bug already proved that path breaks, so the probability of it coming back is not theoretical.

When the test goes first

Two cases, and only these two:

• A bug that can be reproduced. First the test that fails because of the bug, then the fix.
• Logic whose contract is known before the code: a calculation, a parser, a validation, a conversion between formats. You know what goes in and what has to come out, so write that down first.

It goes red for the reason you expect before writing the code: a red caused by an import or a typo proves nothing.

Everything else (exploring an approach, an interface still taking shape, glue code) is written first and tested afterwards, with the criterion above.

Start with the painful case

The happy path is usually covered incidentally by use. What nobody tests is the rest:

• Empty, null, zero, empty string, empty list.
• The maximum and the minimum, and one past each.
• Malformed input, in the wrong format, with accents, very long.
• Duplicates, reverse order, concurrency.
• The external service's error: timeout, 500, a response with a changed shape.

If you are only going to write three tests, make them from here.

A test has to be deterministic

A test that fails one time in twenty is worse than not having it: it teaches the team to retry without looking, and the day it fails for real nobody will believe it.

The four sources of nondeterminism, and what to do about them:

Validate it in the negative

A test you have never seen fail is worthless. It is the most common mistake and the most expensive, because it produces false confidence.

After writing it: deliberately break what it tests, check that the test goes red for the right reason (not because of some other error), and revert. Thirty seconds that tell you whether the safety net exists. If it went red before the code, as in the test-first cases, this is already done.

What to do when it is hard to write

A test that is hard to write is almost never a problem with the test: it is the design giving a warning. If you need to simulate five dependencies to test one function, that function does five things.

Before building the scaffolding, try extracting the pure logic and testing that. The part that talks to the world stays thin and logic-free, and stops needing complicated tests.

Names

The test name is the first thing read when it fails in a log, months later. Make it state the condition and the expectation, not the function name:

• Bad: test_calculate, it("works")
• Good: returns zero when the list is empty, rejects the date if the end is before the start

Before accepting a batch

• Does any fail if I deliberately break what they test?
• Do they pass in random order and in parallel?
• Do they pass on a clean machine, without your cache or your database?
• Is any so slow that people will start skipping them?

Anti-patterns

• Testing the implementation instead of the behaviour. If the test breaks on every refactor without anything observable changing, it is badly written.
• One giant test that tests everything. When it fails you don't know what failed.
• Too many assertions. Checking twenty fields just in case turns any legitimate change into a broken test.
• Doubles that return whatever suits the test. If the double doesn't look like the real response, the test passes and production fails.
• Writing tests to raise a number. Coverage is a signal, not a goal. You can have 90% and test nothing that matters.

Related

• /ship: gates 1 and 5, which are these tests
• /root-cause: before writing the test for a bug, understand the bug
• /explain-plan: the plan's review focus says which failures no test covers