Kill Thresholds Before Launch

A project without a pre-committed kill rule has two failure modes waiting: it dies from panic when noise looks like a signal, or it lives forever as a zombie because no result was ever bad enough. Both are prevented by the same artifact: a threshold written down before launch, sized against noise, with an automated check.

When to use

• Before launching anything whose continuation will later be decided by its results: a live trading or forecasting system, an A/B test, a fake-door test, a product or SaaS probe, a paid-marketing experiment, a hiring-of-a-channel test.
• Before scaling anything up ("double the budget", "go from paper to live") — scaling is a launch and needs its own rule.
• When the agent is asked "should we keep going?" about something already running WITHOUT a pre-committed rule: the first deliverable is the rule (registered now, honestly labeled as post-hoc and therefore weaker), not an opinion about the trajectory.

When NOT to use

• Reversible, cheap, one-shot tasks with no ongoing cost of being wrong.
• Decisions already governed by an existing pre-registered gate (see the preregister skill) — don't stack a second rule that creates ambiguity about which one binds.

The procedure

1. Pick ONE primary metric. If any of five metrics can save the project, nothing can kill it. Secondary metrics may inform the postmortem; only the primary one triggers the rule.
2. Write the rule as a triple: metric + window + action. All three numeric/explicit. "Kill if it does badly" is not a rule. "If cumulative net P&L < -X% at day 90, flatten all positions within 24h and write the postmortem" is a rule. The action must name what physically happens (halt the system, delete the landing page, stop the ad spend, archive the repo) and by when.
3. Size the threshold against noise — this is the step everyone skips. Compute what pure noise would produce over your window. The kill line must sit beyond roughly 2 sigma of the no-edge distribution; otherwise you will kill good projects on bad weeks. Concretely, for a system with daily volatility sigma_d observed over n days:

import math

def t_stat(cum_return, daily_vol, n_days):
    return cum_return / (daily_vol * math.sqrt(n_days))

# The canonical trap: a live system was down -3.2% after 16 days
# with ~1.5% daily volatility. It FELT like a disaster.
print(t_stat(-0.032, 0.015, 16))   # ≈ -0.53  → about half a sigma of noise

# How long does live data take to say anything at all?
# Days needed to distinguish an annualized Sharpe S from zero at 2 sigma:
def days_to_detect(sharpe_annual, z=2.0):
    return 252 * (z / sharpe_annual) ** 2
print(days_to_detect(1.0))  # ≈ 1008 days — ~4 years for a Sharpe-1 edge
print(days_to_detect(2.0))  # ≈ 252 days

Consequence: short-window kill rules for noisy systems must NOT be "is it making money" — they must be drawdown-versus-modeled-distribution rules ("kill if drawdown exceeds the 1st percentile of the backtest's block-bootstrap drawdown distribution for this horizon"), because expected-return estimates carry no information at these horizons.

4. Define BOTH gates — loss and stagnation. The loss gate catches disasters. The stagnation gate ("meh gate") catches zombies: a pre-committed minimum sign of life by a deadline. Every rule needs both (see failure mode c).
5. Write it where future-you cannot ignore it. Commit the rule to the repo next to the launch code; put a calendar event at window-end titled with the rule verbatim; state it to whoever else has standing to hold you to it. A rule that lives only in your head will be renegotiated by the person with the most motivated reasoning available: you, later.
6. Automate the check when possible. A cron job or CI task that computes the metric and emits KILL/CONTINUE beats willpower every time. The owner of the check should not be the person who wants the project to live — a script is the cheapest disinterested party available.
7. When it triggers: execute within the pre-stated deadline (24–48h), no re-litigation. The only permitted override is an exception you listed BEFORE launch (e.g. "exchange outage corrupted 3 days of data"). "But it feels like it's turning around" is never on that list, by construction.
8. Log the outcome either way. Kills produce a 5-line postmortem: rule, trigger value, action taken, one lesson, one thing to never rebuild. Survivals log the margin by which the rule was passed — margins near zero are information for the next window's rule.

Common failure modes

(a) Noise mistaken for signal — in both directions. The -3.2%-in-16-days case above: t ≈ 0.5, literally zero statistical information, yet without a pre-committed rule the operator paused the system on feel. The symmetric error is just as common: +4% in two weeks read as confirmation and used to justify doubling size. Wrong way: letting a sub-1-sigma move — in either direction — drive a decision. Right way: the rule is sized in step 3 so noise CANNOT trigger it; and because it was pre-committed, hope cannot override it when a genuine 3-sigma breach happens. Both properties are load-bearing; a rule with only one is decoration.

(b) Thresholds set after launch. Once you can see the trajectory, any threshold you pick will — with perfect sincerity — land just beyond where you currently are. Post-hoc rules systematically rationalize continuing; that is not a character flaw, it is what knowing the number does to threshold selection. Right way: the rule predates the first data point. If you're already live without one, register the rule now, label it post-hoc in the log, and make the NEXT window's rule the binding one.

(c) Asymmetric rules let zombies live forever. A product probe with a "-50% budget = kill" rule but no stagnation gate gets 2 sales in 60 days — never triggers, never dies, quietly consumes attention and morale for a year. Portfolios reviewed honestly tend to contain several of these: nothing failed, nothing worked, nothing ended. Wrong way: defining kill only for loss. Right way: pair every loss gate with a stagnation gate: "fewer than X units of unambiguous demand by day D = kill", where X is small and D is near.

(d) The rule's owner wants the project to live. Checks run by the project's champion get skipped on exactly the weeks they matter. Right way: automate (step 6) or assign the window-end check to someone/something with no stake.

(e) Metric proliferation. "We'll watch revenue, signups, engagement, NPS, and retention" means the project survives as long as any one of five numbers cooperates. One primary metric, chosen before launch, decides.

Worked examples

A/B fake-door test (product concept). Rule: "If < 5 completed purchases (or purchase-intent clicks) in 30 days from >= 500 unique landing-page visitors: kill the concept, delete the page, write the 5-line postmortem. Stagnation gate: if < 500 visitors reached by day 30, the TEST failed, not the concept — fix distribution and re-run once, then the rule binds." Note the pre-named exception: it distinguishes "no demand" from "no traffic" BEFORE the ambiguity arrives.

Live systematic strategy. Rule: "Primary metric: realized max drawdown. Kill if live drawdown exceeds the 1st percentile of the backtest's block-bootstrap drawdown distribution at the matching horizon. Review (not kill) if 90-day live Sharpe falls below the 5th percentile of the backtest's Deflated-Sharpe-adjusted distribution — Deflated rather than raw, because the backtest was selected from many trials and its raw Sharpe overstates the edge. Action on kill: flatten within 24h, no re-entry without a new pre-registered validation on post-kill data." Do NOT use "negative P&L after N weeks" — step 3 shows weeks of P&L contain no signal for realistic Sharpe levels.

SaaS / paid probe. Rule: "Loss gate: total spend hits the pre-set cap (e.g. 300 of a 500 budget) with zero paying users → kill. Stagnation gate: < 3 activated users OR < $30 MRR by day 60 → kill; additionally, week-over-week signup growth < 1% for 6 consecutive weeks at any point → kill. Action: cancel subscriptions/spend within 48h, archive, postmortem." The 6-week stagnation clause is what prevents the two-sales-zombie.

Checklist

Before launch:

• One primary metric chosen
• Rule written as metric + window + action, all explicit
• Threshold sized against the no-edge noise distribution (~2 sigma), not against feelings
• Both gates present: loss AND stagnation
• Pre-named exceptions listed (and "it feels like it's turning" is not one)
• Rule committed to the repo / calendar / a disinterested witness
• Check automated, or owned by someone without a stake

At window end or on trigger:

• Action executed within the pre-stated deadline, no re-litigation
• 5-line postmortem (kill) or pass-margin note (survive) logged
• Next window's rule set before the next window starts