Falsifier design

A falsifier that cannot fire is not a safety net. It is reassurance — and it is more dangerous than having no falsifier at all, because you stop watching.

The failure

I enabled a flag that relaxed a duplicate-blocking gate. It was a user-visible change, so I did the responsible thing: I declared three metrics that would trigger a rollback if they got worse, recorded their baselines, and shipped.

Later I ran an adversarial check on my own safety net. All three were inert:

I had shipped a user-visible change with no working safety device, while believing I had three.

The one metric that could detect the damage already existed in the payload. I had never looked for it, because I thought I was covered.

The four ways a falsifier is inert

Check every one before you rely on it.

1. Population mismatch

Does the metric's sample overlap what you changed?

A metric built from English-language editorial pairs cannot detect duplicate Korean articles, no matter how good it is. Trace the sample all the way to its source. "It measures duplicates and I'm changing duplicate handling" is not enough — which duplicates?

2. Update latency exceeds your decision window

When does this value actually recompute?

If the metric refreshes on a daily cron and you judge at the next hourly tick, you are reading a pre-change value and calling it "no regression." Write the refresh cadence next to the baseline. If you can't state it, you don't have a falsifier.

3. The floor is already breached

Can you distinguish "worse" from "already bad"?

At 3× the acceptable threshold with a confidence interval wider than the effect you're looking for, the metric can only tell you what you already know. Check statistical power explicitly: with this n and this variance, what size change is detectable? If the answer is "larger than any plausible effect," it's not a falsifier.

4. The denominator moves with the numerator

Does your change alter what gets counted, not just what gets counted as bad?

Any rate whose denominator is "things the gate caught" will be numb to a change that makes the gate catch fewer things. Both terms shrink. Write out the algebra before trusting the ratio.

The procedure

Before shipping, for each declared falsifier, answer in writing:

1. Population — what sample does it draw from, and does that sample contain the thing I'm changing? Trace to source, don't infer from the name.
2. Cadence — when does it recompute, and is that before my judgment point?
3. Power — with the current n, what effect size is detectable? Is the expected damage larger than that?
4. Algebra — does my change move the denominator? Write the ratio both ways.
5. Directionality — is there a mechanism connecting my change to this metric, or just a topical resemblance? Grep the metric's computation for the thing you changed. If it doesn't appear, there's no causal path.

If you cannot answer all five, write down: "shipping without a working falsifier." That sentence is the deliverable. It's fine to ship anyway — what's not fine is shipping while believing you're covered.

Finding a real one

When the declared falsifiers fail, look for a counter that:

• draws from the same population your change touches,
• updates at least as fast as your decision loop,
• is not already saturated, and
• would move only if your change caused harm.

Often it already exists. The best one I found was a shadow counter for a different subsystem that happened to re-check the same population without applying the feature I'd enabled — so it directly counted "cases where the new behavior would have been wrong." Nothing to build.

Look before you build. The payload usually has more in it than you've read.

Related

• verify-before-judging — the same skepticism applied to reading a metric
• blast-radius — the same discipline applied before the change, not after