Hypothesis Falsification

Seek the evidence that would prove you wrong, not the evidence that comforts you. A hypothesis that forbids nothing explains nothing; the value of a theory is what it rules out.

State the hypothesis so it can die

1. Make it falsifiable: phrase it as a prediction with a clear failing observation. "The slowness is the DB" is untestable; "if it's the DB, query latency p99 > 200ms during the spike" can be killed.
2. Enumerate competing hypotheses: the bug has several plausible causes. List them all; the goal is to discriminate, not confirm the first one.
3. Find the discriminating test: design the single observation whose outcome splits the hypothesis space the most (maximize information gain — a test that all hypotheses predict identically is worthless).

The cheapest kill first

• Rank candidate tests by (information gained) / (cost to run). A 30-second log check that eliminates 3 of 5 hypotheses beats a 2-hour reproduction that confirms 1.
• Prefer tests that eliminate over tests that confirm — eliminating a cause is certain; confirming is always provisional (some other cause could produce the same symptom).
• Bisect the hypothesis space (binary search): each test should roughly halve the remaining possibilities.

Adversarial validation (turn it on your own theory)

• Confirmation bias: are you only looking where you expect to find it? Actively seek the observation that contradicts your favored cause.
• Coincidence vs cause: the thing you changed correlates with the symptom — but did you establish the mechanism, or just temporal proximity? Correlation in a single sample is nearly worthless.
• The fix that "worked": did your change fix it, or did the load/condition that triggered it simply pass? Reproduce on demand before claiming a fix.
• Unfalsifiable patching: if every failed prediction spawns a new auxiliary excuse ("it's the DB, except when it's the cache, except..."), the theory is unfalsifiable — discard it.

Non-obvious risks

• Absence of evidence is evidence of absence only if you looked where the evidence would be; a test that couldn't have detected the cause proves nothing.
• The most dangerous hypothesis is the one everyone already believes — it gets the least scrutiny and is tested last.
• Multiple causes can coexist; fixing one and seeing partial improvement can hide a second.

Output structure

Deliver: the falsifiable hypothesis with its killing observation, the full set of competing hypotheses, the discriminating tests ranked by information-per-cost, the result of each test and which hypotheses it eliminated, and the surviving cause confirmed by a reproducible mechanism — not just correlation.