Metrics Design

A metric is an incentive in disguise; people optimize what you measure, including in ways you didn't intend. Design metrics expecting them to be gamed, because they will be.

Separate the three roles a metric can play

1. North-star (what matters): the outcome you actually care about, often lagging and hard to move directly (retention, true user value).
2. Driver (what you can move): leading indicators causally upstream of the north-star. The bet is that moving these moves the north-star — validate that link, don't assume it.
3. Guardrail (what you must not break): metrics that must stay healthy while you optimize the drivers (latency, error rate, cost, trust). Without guardrails, every optimization finds the cheapest harmful shortcut.

Optimizing a driver without a verified link to the north-star, or without guardrails, is how teams hit their numbers while the product dies.

Goodhart-proofing

• "When a measure becomes a target, it ceases to be a good measure." Anticipate the cheapest way to move the number without creating the underlying value, and add a guardrail or paired metric that makes the shortcut visible.
• Pair metrics that trade off: speed + quality, volume + retention, activation + long-term engagement. A single metric is always gameable; a tension pair is harder to cheat.
• Prefer ratios and rates over raw counts (counts grow with traffic and flatter without improvement), and distributions over averages (an average hides the harmed tail).

Adversarial validation

• Game it yourself: if your bonus depended on this metric, how would you move it without doing your job? That exploit is your design flaw.
• Vanity check: does this number go up just because the business grew, regardless of whether we improved? (Total signups vs activation rate.)
• Proxy gap: the metric measures a proxy — under what conditions does the proxy diverge from the real outcome? (e.g., "time on site" up because the UI got confusing.)
• Survivorship: does the metric only count users who stuck around, hiding the ones the change drove away?
• Lagging blindspot: if the metric only moves months later, what leading signal tells you sooner that you're wrong?

Non-obvious risks

• Every metric you publish becomes a target; publishing too many dilutes focus, too few invites tunnel vision.
• Local metrics drive local optimization that harms the global outcome (team A's metric improves by dumping work on team B).
• A metric with no owner and no decision attached is measurement theater — costs collection effort, changes nothing.

Output structure

Deliver: the north-star with its (validated, not assumed) causal link to the chosen drivers, the guardrails that bound the optimization, the paired/tension metrics that resist gaming, the self-run gaming exploit and its mitigation, and the decision each metric is meant to inform.