diagnosing-bugs

A discipline for hard bugs. Skip a phase only when you can say why.

The whole skill is Phase 1: get a loop that goes red on this bug. Everything after is mechanical once you have it. If you catch yourself reading code to form a theory before that loop exists, stop — jumping to a hypothesis without a red-capable loop is the exact failure this prevents.

For trivial bugs with an obvious fix and an obvious test, skip this and just write the failing test — this skill is for the bugs that resist.

Phase 1 — Build a feedback loop

A tight loop is fast, deterministic, and goes red on this bug. Build one and the bug is 90% solved; bisection, hypothesis-testing, and instrumentation all just consume it. Be aggressive and creative here — spend disproportionate effort.

Ways to construct one — try roughly in this order

1. Failing test at the seam — dart test path/to/test.dart or flutter test, unit or widget, at whatever seam reaches the bug. Tightest loop there is.
2. fakeAsync harness — for timer, microtask, and timing-dependent bugs (most flakes). Pin the clock via options.clock, elapse time deterministically.
3. testWidgets harness — for Flutter UI bugs: pump the widget tree, drive the gesture, assert on the tree (e.g. a hit-test or navigation regression).
4. Integration test — flutter test integration_test/... when the bug crosses the native (JNI/FFI) boundary, which cannot be faked (see packages/flutter/AGENTS.md).
5. Replay a captured payload — save a real envelope / event / span / network payload to disk and replay it through the code path in isolation.
6. Throwaway harness — a minimal void main() that exercises the bug path with one call and mocked deps.
7. Stress/repetition loop — for non-deterministic bugs: run the trigger 100×, parallelise, narrow timing windows, inject delays. The goal is a higher reproduction rate, not a clean repro.
8. Differential loop — run the same input through two versions or two configs (e.g. before/after a dependency bump) and diff the output. For regressions that "appeared after X".
9. Bisection harness — if it appeared between two commits, script the red/green check and git bisect run it.

Tighten the loop

Once you have a loop, treat it as a product and tighten it:

• Faster — narrow the test scope, skip unrelated init.
• Sharper — assert the exact symptom the user described, not "didn't crash".
• More deterministic — pin the clock (options.clock), use fakeAsync, seed any randomness, avoid real network/filesystem (per test-guidelines). A 2-second deterministic loop beats a 30-second flaky one.

For non-deterministic bugs the goal is a high enough reproduction rate to debug against — a 50% flake is debuggable, 1% is not. Keep raising the rate.

Completion criterion

Phase 1 is done when you can name one command — a test invocation or script path — that you have already run at least once (paste the invocation and its output), and that is:

• Red-capable — drives the actual bug path and asserts the user's exact symptom, so it goes red now and green once fixed
• Deterministic — same verdict every run (flaky bugs: a pinned, high reproduction rate)
• Fast — seconds, not minutes

No red-capable command, no Phase 2.

When you genuinely cannot build a loop

Say so explicitly, list what you tried, and ask the user for: access to an environment that reproduces it, a captured artifact (envelope dump, log, CI run, screen recording with timestamps), or permission for temporary instrumentation. Do not hypothesise without a loop.

Phase 2 — Reproduce and minimise

Run the loop, watch it go red. Confirm it produces the failure mode the user described — not a nearby one (wrong bug = wrong fix). Then shrink the repro to the smallest scenario that still goes red: cut inputs, callers, config, and steps one at a time, re-running after each cut. Done when every remaining element is load-bearing — removing any one turns it green. The minimal repro shrinks the hypothesis space and becomes the regression test.

Phase 3 — Hypothesise

Generate 3–5 ranked, falsifiable hypotheses before testing any — a single hypothesis anchors you on the first plausible idea. Each must state its prediction: "If X is the cause, then changing Y makes the bug vanish." If you can't state the prediction, it's a vibe — sharpen or discard it. Show the ranked list to the user before testing; they often re-rank it instantly ("we just changed #3"). Don't block on it if they're away.

Phase 4 — Instrument

Each probe maps to one prediction from Phase 3. Change one variable at a time.

• Prefer a debugger / breakpoint over logs where the env supports it. One breakpoint beats ten logs.
• Otherwise add targeted logs at the boundaries that distinguish hypotheses — never "log everything and grep".
• Tag every debug log with a unique prefix like [DEBUG-a4f2] so cleanup is one grep.
• Performance regressions: logs are the wrong tool. Establish a baseline measurement (Stopwatch, a timing harness, DevTools, the observatory), then bisect. Measure first, fix second.

Phase 5 — Fix and regression test

Write the regression test before the fix — but only at a correct seam, one that exercises the real bug pattern as it occurs at the call site (per test-guidelines). A too-shallow seam (a unit test that can't replicate the chain that triggered it) gives false confidence. If no correct seam exists, that is itself the finding — note it; the architecture is preventing the bug from being locked down.

With a correct seam: turn the minimised repro into a failing test, watch it fail, apply the fix, watch it pass, then re-run the Phase 1 loop against the original un-minimised scenario.

Phase 6 — Cleanup and post-mortem

Before declaring done:

• Original repro no longer reproduces (re-run the Phase 1 loop)
• Regression test passes (or the absence of a seam is documented)
• All [DEBUG-…] instrumentation removed (grep the prefix)
• Throwaway harnesses deleted
• The correct hypothesis is stated in the commit / PR message, so the next debugger learns

Then ask: what would have prevented this bug? If the answer is architectural — no good test seam, tangled callers, a shallow module hiding the real bug behind it — hand off to the design-first skill with the specifics. Make that call after the fix is in, when you know the most.