Investigating bugs

Avoid "guess-and-check" coding. Always identify the root cause before making changes.

THE IRON LAW: NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST.

Violating the letter of the rules is violating the spirit of the rules.

REQUIRED PREREQUISITE: You must have already completed slow-powers:working-in-isolation — debug from an isolated workspace.

REQUIRED BACKGROUND: You must understand slow-powers:working-with-tdd — Phase 4 captures the bug with a failing test before fixing it.

Phase 1: root cause investigation

Complete these steps before changing any code:

1. Read error messages and stack traces: Read every line of the error. Note the exact file, line number, and error codes.
2. Reproduce consistently: Identify the exact steps, inputs, or environment needed to trigger the bug. If it cannot be reproduced, gather more logs instead of guessing.
   • For flaky tests (pass sometimes, fail under load or only in CI), find the non-determinism before changing anything — don't rerun until it goes green. Arbitrary sleep/timeout delays are one common cause (wait on the actual condition, not a guessed duration — see condition-based waiting); a dependency called a non-deterministic number of times or in a non-deterministic order is another. Read diagnosing flaky tests for the general diagnostic method and cause catalog.
3. Check recent changes: Run git diff. Analyze recent commits, dependency additions, or configuration changes.
4. Gather evidence in multi-component systems:
   • Log inputs and outputs at every component boundary.
   • Instrument the layers step by step (for example, workflow → build script → runtime → database) to pinpoint exactly where the state breaks.
5. Trace data flow: Trace variables backward from the failure point to their source. Fix the bug at the source, not the symptom.
   • When manual tracing dead-ends, instrument the suspect operation: log the key inputs, relevant environment, and a captured stack trace (new Error().stack) just before it runs. In tests, write to stderr — a logger may be suppressed. Read the captured stack to find the original caller, then remove the instrumentation.

Phase 2: pattern analysis

1. Find working examples: Search the codebase for similar logic that functions correctly.
2. Compare implementations: Identify every difference between the working version and the failing version. Do not assume "that difference doesn't matter."
3. Verify dependencies and configuration: Ensure all required modules, configuration, and environment variables are present and correct.

Phase 3: hypothesis and testing

1. Formulate a single hypothesis: Write down a clear statement: "I think X is the root cause because Y."
2. Test minimally: Make the smallest possible change to verify the hypothesis (for example, add a log or change one value).
3. Verify and reevaluate: Did the test prove your hypothesis?
   • Yes: Proceed to Phase 4.
   • No: Revert the test change completely and formulate a new hypothesis. Never stack guess-on-guess.

Phase 4: implementation and verification

1. Write a failing test case: Create an automated test or simple script that consistently triggers the bug. Verify it fails.
2. Implement the fix: Make a single, targeted change that directly addresses the root cause. Do not bundle unrelated refactoring.
3. Verify the fix: Run the test suite. Ensure the new test passes and no regressions are introduced.
4. Apply the three-fix limit (architectural check):
   • If you attempt three separate fixes and the bug remains: STOP.
   • This is a strong signal that the issue is architectural (e.g., wrong model assumptions, coupled state, race conditions).
   • Re-evaluate the system architecture and discuss the approach with the user before attempting a fourth patch.

Common rationalizations

Red flags — STOP and reset

• Writing a fix before reproducing the bug or reading the full stack trace
• "Let's just try changing X to see if it works"
• Stacking multiple speculative fixes on top of each other
• Claiming a bug is fixed without running the verification test suite
• Each "fix" only shifts the bug to a new location

All of these mean: STOP. Revert changes. Return to Phase 1.