Test-Driven Development

Invariant Principles

1. Failure Proves Testing - Test passing immediately proves nothing. Only watching failure proves test detects what it claims.
2. Order Creates Trust - Tests-first answer "what should this do?" Tests-after answer "what does this do?" Fundamentally different questions.
3. Minimal Sufficiency - Write exactly enough code to pass. YAGNI violations compound into untested complexity.
4. Deletion Over Adaptation - Code written before tests is contaminated. Keeping "as reference" means testing after. Delete means delete.

Violating the letter of the rules is violating the spirit of the rules.

Inputs

Outputs

When to Use

Always: new features, bug fixes, refactoring, behavior changes.

Exceptions (ask human partner; no human available? default: apply TDD):

• Throwaway prototypes
• Generated code
• Configuration files

Thinking "skip TDD just this once"? Stop. That's rationalization.

The Iron Law

NO PRODUCTION CODE WITHOUT FAILING TEST FIRST

Code before test? Delete. Start over. No "reference," no "adapting," no looking at it.

Reasoning Schema

Red-Green-Refactor

RED: Write Failing Test

One behavior. Clear name. Real code (mocks only if unavoidable; unavoidable = external I/O, time, hardware — not laziness).

const result = await retryOperation(operation);

expect(result).toBe('success'); expect(attempts).toBe(3); });

Clear name, tests real behavior, one thing
</Good>

<Bad>
```typescript
test('retry works', async () => {
  const mock = jest.fn()
    .mockRejectedValueOnce(new Error())
    .mockRejectedValueOnce(new Error())
    .mockResolvedValueOnce('success');
  await retryOperation(mock);
  expect(mock).toHaveBeenCalledTimes(3);
});

Vague name, tests mock not code </Bad>

Verify RED: Watch It Fail

MANDATORY. Never skip.

npm test path/to/test.test.ts

Confirm:

• Fails (not errors)
• Failure message expected
• Fails because feature missing (not typos)

Test passes? Testing existing behavior. Fix test. Test errors? Fix error, re-run until it fails correctly.

GREEN: Minimal Code

Simplest code to pass. No features, no refactoring, no "improvements."

Load the enforcing-code-quality skill before writing this code. "Minimal" bounds SCOPE, not quality: the prohibited patterns (any types, blanket try-catch, resource leaks, unvalidated non-null assertions) are not licensed by the fact that a test is currently red. A shortcut taken here survives REFACTOR, because REFACTOR only cleans what it recognizes as duplication.

Verify GREEN: Watch It Pass

MANDATORY.

npm test path/to/test.test.ts

Confirm:

• Test passes
• Other tests still pass
• Output pristine (no errors, warnings)

Test fails? Fix code, not test. Other tests fail? Fix now.

REFACTOR: Clean Up

After green only. Remove duplication, improve names, extract helpers. Keep tests green. Don't add behavior.

Complete when: duplication removed, names clear, all tests still green, no new behavior added.

Repeat

Next failing test for next feature. The cycle continues until all behavior is implemented.

Good Tests

Assertion Quality

Every assertion MUST assert exact equality against the COMPLETE expected output. This applies to ALL output -- static, dynamic, or partially dynamic. There are no categories of output exempt from this rule.

# CORRECT: exact equality on complete static output
assert result == expected_complete_output

# CORRECT: exact equality with dynamically constructed expected value
assert message == f"Today's date is {datetime.date.today().isoformat()}"

# BANNED: partial assertion on any output -- dynamic content is no excuse
assert "substring" in result          # Hides structural errors, missing content, extra garbage
assert datetime.date.today().isoformat() in message  # Dynamic value is no excuse for partial check
assert "foo" in result and "bar" in result  # Still partial, still BANNED
assert len(result) > 0                # Meaningless

# BANNED: mock.ANY hides argument values
mock_fn.assert_called_with(mock.ANY, mock.ANY)

ALL output demands complete verification: writers, serializers, formatters, code generators, query builders, template renderers, config builders, functions with dynamic content. For multi-line output, use triple-quoted strings or dedent helpers. Output length is NEVER a justification for partial assertions.

When output contains dynamic values (timestamps, derived strings), construct the complete expected value using the same logic, then assert ==. Do not assert partial membership of the dynamic value.

For mock assertions: assert EVERY call with ALL args, verify call count. mock.ANY is BANNED -- construct the expected argument dynamically if needed.

Normalization is LAST RESORT only -- for truly unknowable values (random UUIDs, OS-assigned PIDs, memory addresses). Never use normalization to avoid constructing a complete expected value. </CRITICAL>

Tests must validate CONTENT, not just EXISTENCE. Every assertion must answer: "If the value was garbage, would this catch it?"

Reference: Read patterns/assertion-quality-standard.md for the full Assertion Strength Ladder, Full Assertion Principle, Bare Substring Problem analysis, and Broken Implementation Test. Every assertion must be Level 4+ on the ladder.

Rules

Pychoir Matchers

Pychoir matchers (including custom subclasses) are the ONE exception to the mock.ANY ban. Use them ONLY when the value genuinely cannot be known ahead of time (timestamps, UUIDs, auto-incremented IDs). Each use requires a stated justification in a comment.

# GOOD: UUID genuinely unknowable, justification stated
from pychoir import IsInstance
assert result == {"id": IsInstance(str), "name": "expected_name"}  # id is server-generated UUID

# BAD: Using matcher to avoid computing expected value
assert result == {"count": IsInstance(int), "items": IsInstance(list)}  # Lazy - compute the expected values

ESCAPE Analysis (Mandatory Per Test Function)

Before marking ANY test function complete, fill in this template:

ESCAPE: [test_function_name]
  CLAIM: What does this test claim to verify?
  PATH:  What code actually executes?
  CHECK: What do the assertions verify?
  MUTATION: For each assertion, name the specific production code mutation it catches.
  ESCAPE: What specific broken implementation would still pass this test?
  IMPACT: What breaks in production if that broken implementation ships?

The MUTATION field is a forcing function: for each assertion in the test, you must name a specific, plausible production code change that would cause that assertion to fail. If you cannot name one, the assertion is too weak and must be strengthened before proceeding. This is the Broken Implementation Test from patterns/assertion-quality-standard.md.

The ESCAPE field must describe a SPECIFIC broken implementation, not a generic statement. "Nothing reasonable" IS valid when justified, but requires explanation of why the assertions are comprehensive enough.

Red flags in ESCAPE analysis:

• ESCAPE field says "none" or "nothing" without justification
• ESCAPE field describes something the test DOES catch (means you didn't think hard enough)
• ESCAPE field is copy-pasted between tests (each test has unique escape paths)

Test Strength Verification (mechanical pre-commit grep)

ESCAPE Analysis is the rigorous version. This is the cheap mechanical version that catches the most common green-mirage patterns at commit time. Run BEFORE any per-task commit:

# 1. Banned substring assertions on serialized output
rg -n 'expect.*stringContaining.*JSON|expect.*stringContaining.*\{' <test_files>
# Replace with: JSON.parse(...) + exact deep equality (toEqual)

# 2. Tautological / dead assertions
rg -n 'expect\(true\)\.toBe\(true\)|expect\(1\)\.toBe\(1\)|assert True$|assert 1$' <test_files>
# Replace with: behavioral assertion that can actually fail

# 3. Mock call without argument verification (catches "called X but with what?")
rg -n 'toHaveBeenCalled\(\)$|assert_called\(\)$|called_once\(\)$' <test_files>
# Strengthen with: toHaveBeenCalledWith(...) / assert_called_with(...) / called_once_with(...)

# 4. Existence-only smoke tests (typeof === "function")
rg -n 'typeof.*===.*function|isinstance.*function|callable\(' <test_files>
# Replace with: actually call the thing and assert behavior

# 5. Side-effect-free "happy path" tests for stateful operations
#    (e.g., "forms a coalition" that asserts return value but never
#    inspects the Redis hash that should have been written)
#    — no greppable signature; require by checklist:
#    For every test of a method that mutates external state, at least one
#    assertion MUST inspect the post-state, not just the return value.

If any grep returns hits: strengthen the assertion BEFORE the next gate. A passing test with these patterns is a known green-mirage source (see auditing-green-mirage skill for the catalog).

Mock fidelity rule: If your mock for an external system (Redis Lua, SQL, HTTP API) does not match the real system's contract, your test passes are lies. When integration infrastructure is available (Phase 4.0 Environment Gate confirmed it), at least one integration test per module MUST exercise the real system. Mocks for evalLua, set, xadd, etc. are convenience aids, not coverage.

Evidence Requirements

Anti-Patterns

• Code before test
• Test passes immediately (without watching it fail)
• Can't explain why test failed
• "Just this once" / "already manually tested"
• "Keep as reference" / "adapt existing"
• "Tests after achieve same goals"
• "TDD is dogmatic, being pragmatic"
• "It's about spirit not ritual"
• "Already spent X hours, deleting is wasteful"
• "This is different because..."
• Tests added "later" / tests after implementation
• Rationalizing "I already manually tested it"
• "Need to explore first" (throw away exploration, start with TDD)</FORBIDDEN>

Common Rationalizations

Self-Check

Before marking complete:

• Every function has test
• Watched each test fail before implementing
• Failed for expected reason (feature missing, not typo)
• Wrote minimal code to pass
• All tests pass, output pristine
• Tests use real code (mocks only if unavoidable)
• Edge cases and errors covered
• Every assertion validates CONTENT, not just existence/count
• Every assertion is Level 4+ on the Assertion Strength Ladder (patterns/assertion-quality-standard.md)
• ESCAPE analysis completed for every test function (including MUTATION field)
• Every assertion has a named mutation that would cause it to fail
• No mock.ANY in assertions (use pychoir matchers with justification comment instead)
• Every mock call asserted with ALL arguments; call count verified
• No len() > 0 or len() == N without content verification
• No bare substring checks on string output (dynamic content is no excuse -- construct full expected value)

If ANY unchecked: Skipped TDD. Start over.

When Stuck

Bug Fix Pattern

Bug: Empty email accepted

RED

test('rejects empty email', async () => {
  const result = await submitForm({ email: '' });
  expect(result.error).toBe('Email required');
});

Verify RED

$ npm test
FAIL: expected 'Email required', got undefined

GREEN

function submitForm(data: FormData) {
  if (!data.email?.trim()) {
    return { error: 'Email required' };
  }
  // ...existing logic
}

Verify GREEN

$ npm test
PASS

REFACTOR Extract validation for multiple fields if needed.

Never fix bugs without a test.

Testing Anti-Patterns

When adding mocks or test utilities, avoid common pitfalls:

Test Speed & Scope

Fast tests enable tight red-green-refactor cycles. Slow tests break flow.

• Isolate expensive resources: Mock GPU, network, and DB calls in unit tests. Real resources belong in integration tests only.
• Smallest possible inputs: 4x4 matrices, not 1024x1024. Save large inputs for performance/integration tests.
• Never sleep in tests: Poll with short intervals, or mock the time-dependent component.
• Lightweight fixtures: If a fixture takes longer than the test itself, it is too heavy for a unit test.

Apply marks proactively when writing new tests. A test that calls a GPU kernel is a GPU test even if it is fast today. Common marks: slow, gpu/hardware, network/external, integration, smoke.

Scope test runs to changes: if src/auth/login.py changed, run tests/test_login.py, not the entire suite. Run the full suite once at the end of a work unit, not after every edit.

Final Rule

Production code -> test exists and failed first
Otherwise -> not TDD

No exceptions without your human partner's permission.

<FINAL_EMPHASIS> The test must fail first. You must watch it fail. The code must be minimal. There are no shortcuts. Every rationalization is a trap. Delete code written before tests. Start over with TDD. </FINAL_EMPHASIS>