investigate-metric
Skill4 starsChanged 8 days agoMITView on GitHub
Structured anomaly investigation and root-cause analysis for an underperforming or anomalous metric — verify the data, localise the change, correlate with business events, test hypotheses, and conclude with confidence levels. Use when a metric looks wrong, dropped, or spiked.
3 other projects use a version of this.
Adopted by
Most recently taken up by
- patinaproject/skillstheir own version, first seen Oct 15, 2026
- PostHog/ai-plugintheir own version, first seen Sep 25, 2026
- PostHog/skillstheir own version, first seen Sep 13, 2026
Other versions3 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 3
- PostHog/ai-plugin★ 100same name
- PostHog/skills★ 80same name
- patinaproject/skills★ 1same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository21 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- analyzeSkill
- build-dashboardSkill
- build-modelSkill
- build-pipelineSkill
- causal-analysisSkill
- connect-dataSkill
- define-kpisSkill
- evalSkill
- experimentSkill
- forecastSkill
- health-checkSkill
- make-deliverableSkill
- powerbiSkill
- privacy-reviewSkill
- research-domainSkill
- scorecardSkill
- setup-teamSkill
- tableauSkill
- triageSkill
- upgradeSkill
The file
We split this skill into the rules it gives an agent
1 rule. 0 are common, 0 are distinctive, and 1 is very specific to this project.
0
Common
Most of the 3 edited versions we read keep these.
0
Distinctive
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 30
Close the loop: Update the relevant scorecard note from "under investigation" to the verdict.
“Common” is measured: kept by at least 6 in 10 of the 3 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Investigate Metric — Anomaly & Root Cause
Owner: performance-monitor, pulling in bi-analyst for deep decomposition. Args name the metric and symptom, e.g. /investigate-metric signups down 20% this week.
Procedure — strictly in this order
- Define the anomaly precisely. Metric (catalog definition), observed value, expected value/range, period, where it was observed (which dashboard/scorecard — different surfaces can disagree). Quantify the gap.
- Verify it's real (data before business). Check, in order: source pipeline freshness and row counts for the period; recent changes to tracking/instrumentation; recent changes to the metric definition or its upstream models (
knowledge/decision-log.md, git history); duplicates/deduplication changes; timezone or date-boundary artifacts. ~half of metric alerts are data problems — rule these out first. If it's a data problem: fix or route to data-engineer, correct the record wherever the wrong number was shown, done. - Localise. Decompose the change across every meaningful dimension: segment, channel, geography, product, plan/tier, new-vs-returning, device/platform. Find concentration ("the entire drop is paid-search signups in region X"). Run mix-shift analysis to separate rate changes from composition changes — a flat or rising topline can mask every subgroup falling (Simpson's paradox), so stratify before trusting the aggregate. Decompose, don't eyeball: for a rate or average metric, split the change exactly into a mix effect (Σ change in each segment's share × its prior rate) and a rate effect (Σ each segment's new share × the change in its rate) — the two sum to the total move, and they tell different stories. For an additive metric, rank segments by contribution to the change, not by their own percentage move (a small segment can double and not matter). For a metric that is a product (revenue = customers × revenue per customer), walk down the metric tree in
knowledge/metrics-catalog.mdone driver at a time. Also sanity-check whether an extreme prior period is simply regressing to the mean rather than a true change. (analytics.mdPart 1.) - Correlate in time. Align the change point with the business change ledger: releases/deploys, pricing changes, campaigns starting/ending, site/app changes, outages, seasonality and holidays, external/market events. Sources: {{CHANGE_LEDGER_SOURCES e.g. "deploy log, marketing calendar — ask the user what changed if no ledger exists"}}. Build a simple timeline.
- Hypothesise & test. Write down 2–4 ranked candidate causes. A change that lines up in time with the anomaly is a suspect, not a culprit: to claim it caused the move, the test has to rule out what else changed (
analytics.mdPart 4). Where a comparison group exists that the change did not reach, use it — that is a difference-in-differences, and/causal-analysisruns it properly. For each, derive a testable implication ("if it's the checkout change, mobile should be hit harder") and test it against the data. Record which hypotheses survive and which are ruled out. - Conclude. Write
analyses/YYYY-MM-DD-rca-<metric>/FINDINGS.md:- Verdict: most likely cause + confidence (high/medium/low) — or "unresolved; ruled out A, B, C"
- Evidence trail (the localisation and hypothesis tests)
- Impact: quantified cost of the anomaly to date and per period if it continues
- Recommended action and what evidence would confirm/deny the verdict
- Whether monitoring should change (new alert, new threshold, paired guardrail metric)
- The provenance footer (
standards/reporting-standards.md). A verdict that will reach leadership goes toanalysis-reviewerfirst.
- Close the loop. Update the relevant scorecard note from "under investigation" to the verdict. Log the cause in
knowledge/decision-log.md(the change ledger gets smarter every investigation). If the cause implies urgent business impact, lead the report with that, prominently.
Created by Colin Beck — https://www.linkedin.com/in/beckcolin/
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

