Systematic Debugging

When to Use

• Any bug, test failure, or unexpected behavior — before proposing fixes
• Build failures, runtime errors, or flaky tests
• NOT for: feature design (brainstorming), code review (providing-code-review)

Core principle: ALWAYS find root cause before attempting fixes.

Wrong skill? Feature design → brainstorming. Code review → providing-code-review.

NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST

If you haven't completed Phase 1, you cannot propose fixes.

Human Partner Signals You're Doing It Wrong

If the human says any of these, STOP immediately:

• "You already tried that"
• "That's not the problem"
• "We've been going in circles"
• "Stop and think about this differently"
• "You're making this too complicated"
• "Let's step back"

Do not defend the current approach. Do not continue it. Acknowledge what they said, ask what they're seeing that you're missing, and restart the investigation from scratch.

The Four Phases

Phase 1: Root Cause Investigation

BEFORE attempting ANY fix:

1. Read Error Messages Carefully — stack traces completely, note line numbers, file paths, error codes
2. Reproduce Consistently — exact steps, every time? If not reproducible, gather more data, don't guess
3. Check Recent Changes — git diff, recent commits, new dependencies, config changes
4. Gather Evidence in Multi-Component Systems — for each component boundary: log what enters, log what exits, verify env/config propagation. Run once to find WHERE it breaks.
5. Trace Data Flow — where does bad value originate? Trace up the call stack to find the source. Fix at source, not symptom. See root-cause-tracing.md for complete technique.

Phase 2: Pattern Analysis

1. Find working examples in same codebase — what works that's similar to what's broken?
2. Compare against references — read reference implementations COMPLETELY, don't skim
3. Identify differences — list every difference, don't assume "that can't matter"
4. Understand dependencies — components, settings, config, environment, assumptions

Phase 3: Hypothesis and Testing

1. Form single hypothesis — "I think X is the root cause because Y"
2. Test minimally — smallest possible change, one variable at a time
3. Verify — worked → Phase 4. Didn't work → new hypothesis, don't stack fixes.

Phase 4: Implementation

1. Create failing test case — use test-driven-development skill
2. Implement single fix — ONE change, no "while I'm here" improvements
3. Verify — test passes, no other tests broken
4. Record the refactoring disposition as a Refactoring-Disposition: line in the fix commit message (or the investigation-state note, if the session has one) — same vocabulary as hotfix-charter: REFACTOR_NOW (the refactor is the fix), PARK (you saw a cleanup worth doing and deliberately deferred it — name what), or NONE (nothing structural worth refactoring) — plus one concrete reason. Default NONE for a clean fix; PARK only when you actually spotted deferrable structure. This is a wrap-up note, NOT license for a "while I'm here" edit (item 2 still governs: ONE change).
5. If 2+ fixes failed — STOP. Question the architecture. Each fix revealing new problems in different places = wrong architecture, not wrong fix. Discuss with human before continuing.

Red Flags — STOP, Return to Phase 1

• "Quick fix for now, investigate later"
• "Just try changing X and see"
• Proposing solutions before tracing data flow
• "One more fix attempt" after 2+ failures

Recovery: After 2+ Failed Fix Attempts

When fixes keep failing, the problem is usually misdiagnosed. Don't try a third fix — escalate. If 2 or more distinct approaches have failed, the underlying architecture or assumptions are wrong. Stop adding fixes.

1. Invoke think-twice — verbalize what you've tried and why each failed
2. Question the layer — are you fixing the right component? Check one layer up and one layer down
3. Check assumptions — list every assumption you've made. Test the least-certain one first. Write down what you assumed was true at the start and check each against evidence; the broken assumption is likely the root cause.
4. Ask the human — "I've tried X and Y, both failed because Z. My current hypothesis is W — does that match your understanding?"

Common Rationalizations

Failure Modes

Supporting Techniques

• root-cause-tracing.md — trace bugs backward through call stack
• defense-in-depth.md — add validation at multiple layers
• condition-based-waiting.md — replace arbitrary timeouts with condition polling

Companion Skills

• investigation-state — persist debugging context across sessions for multi-day bugs
• think-twice — dispatch fresh sub-agent when stuck in a hypothesis loop
• adversarial-search — search for the WRONG value when symptoms contradict expectations
• receiving-code-review — responding to review feedback
• failure-autopsy — post-mortem on failed approaches
• hotfix-charter — the mechanically-enforced home of the REFACTOR_NOW/PARK/NONE disposition vocabulary (Phase 4 item 4)