hype-check
Skill0 starsChanged 4 months agoMITView on GitHub
Separate "X is happening / X exists" (prevalence) from "X causes Y by magnitude Z" (measured impact), and surface the missing denominator, the absent counterfactual, and the confound behind a scary or exciting claim. Use whenever reading a viral statistic, a trend piece, a threat narrative, a marketing claim, or a "this changes everything" headline; before repeating or acting on one. Triggers on "is this as big a deal as they say," "X is exploding/surging," fear-driven or FOMO-driven claims, "N% of people now do X," and any number that feels designed to alarm or impress. If a claim moves fast and feels huge, run it through this first.
In this repository11 other files it gives its agents.
- analogy-checkSkill
- change-my-mindSkill
- disaggregateSkill
- disparate-impact-auditSkill
- grade-the-evidenceSkill
- prose-surgeonSkill
- protect-the-skillSkill
- scenario-branchSkill
- steelmanSkill
- validate-the-claimSkill
- value-framesSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 29
Every big claim quietly conflates two different measurements.
- Distinctive: line 55
Before crediting X, list what else changed at the same time.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Hype Check
Most misleading claims aren't false — they're true about the wrong thing. "77% of voters encountered a deepfake" is true and tells you almost nothing about whether deepfakes changed a single vote (independent fact-checkers found under 1% of flagged election misinformation was actually AI-generated, and a large field experiment found AI microtargeting no more persuasive than generic messaging). The claim measured prevalence and let you infer impact. This skill pulls those two apart and asks the three questions hype is built to skip: what's the denominator, where's the counterfactual, and what else could explain this?
The core split: prevalence is not impact
Every big claim quietly conflates two different measurements. Name which one the claim actually supports:
- Prevalence / existence — "X is happening," "X exists," "N% encountered X," "incidents of X rose." Usually the easy thing to measure, and usually all the evidence actually shows.
- Measured causal impact — "X caused Y," "X changed the outcome by magnitude Z." The hard thing, requiring a counterfactual, and usually absent.
A claim that establishes prevalence and lets you infer impact is the single most common move in hype. When you see a scary/exciting stat, first ask: does this measure that the thing happens, or that it matters? They are not the same evidence, and the second is the one you actually care about.
The three questions hype skips
1. Where's the denominator?
Big raw numbers are meaningless without the base they're drawn from.
- "5 documented cases of X preceding harm" — out of how many total users? A numerator with no denominator can't be a rate, and can't tell you if X is rare or common.
- "Incidents up 1,400%" — from what base? A jump from 2 to 30 is a huge percentage and a tiny absolute. Always convert to rate-per-exposed and compare to a baseline.
- Missing-denominator claims are often unknowable rates dressed as alarming ones. Say when the denominator genuinely doesn't exist — that's a finding.
2. Where's the counterfactual?
Impact is a comparison to what would have happened otherwise. If there's no comparison, there's no impact claim.
- "Users of X had outcome Y" — compared to non-users? Matched how? Without a control, Y might have happened anyway.
- "Sales rose after we shipped X" — versus a world without X, or just versus last quarter (when sales were rising regardless)? Two numbers moving requires no causal story.
3. What's the confound?
Before crediting X, list what else changed at the same time.
- Correlated trends: news layoffs rose and AI content farms grew — shared upstream drivers (ad economics), not necessarily one causing the other. Treat as confounded, not isolated.
- Reverse causation: "AI-leader firms outperform" — or do well-run firms both adopt AI and outperform for unrelated reasons?
- Selection: who's in the sample, and did they self-select in a way that produces the result?
The forecasting-hype variant
A special case: "N% of jobs/tasks at risk" style projections. Exposure is not outcome. A famous 2013 forecast put 47% of US jobs at high automation risk within a decade; unemployment then fell to multi-decade lows, and the occupations rated highest-risk sometimes grew. Task-level exposure measures technical susceptibility, not job loss — the outcome depends on bargaining power, retraining, and demand. When you see "X% exposed," refuse to read it as "X% will be lost." (For the deeper version, see the protect-the-skill and analogy-check skills.)
Procedure
- State the claim and what it feels like it proves.
- Classify: does the evidence establish prevalence, or measured impact? (Usually prevalence.)
- Denominator: is there a base rate? If a raw count, convert to a rate; if none exists, say so.
- Counterfactual: is there a comparison to "otherwise"? If not, the impact claim is unsupported.
- Confounds: list what else could produce this (correlated trend, reverse causation, selection).
- Restate what the claim actually supports — usually narrower than the headline — and what evidence would be needed to support the strong version.
Output template
CLAIM: <as stated> | FEELS LIKE IT PROVES: <the strong reading>
MEASURES: prevalence | measured impact (which does the evidence actually show?)
DENOMINATOR: <base rate present? raw count → rate? or genuinely unknowable?>
COUNTERFACTUAL: <compared to what? present or absent?>
CONFOUNDS: <correlated trend | reverse causation | selection — what else fits?>
WHAT IT ACTUALLY SUPPORTS: <the honest, usually-narrower claim>
TO SUPPORT THE STRONG VERSION, YOU'D NEED: <the missing evidence>Example (compressed)
Claim: "AI companions are causing the teen loneliness epidemic — 70% of teens have used one!"
MEASURES: prevalence. "70% have tried one" says usage is common. It says nothing
about causing loneliness.
DENOMINATOR: the 70% is fine as a prevalence rate — but there's no rate linking
use to onset of loneliness.
COUNTERFACTUAL: absent. No comparison of matched users vs. non-users over time.
And teen sadness was already trending up for years before companions existed —
adoption is a single recent point on a pre-existing curve.
CONFOUNDS: reverse causation is live and evidence-backed — lonelier teens seek
companions, so use may be a *symptom*, not a cause. Correlational data even
shows heavier use tracking more loneliness, which fits both directions.
WHAT IT ACTUALLY SUPPORTS: AI companion use is common among teens, and heavier
use correlates with loneliness — direction unresolved.
TO SUPPORT "CAUSING THE EPIDEMIC": you'd need longitudinal/experimental evidence
isolating companion use from pre-existing loneliness. It doesn't exist yet.Gotchas
- Deflating hype is not denying the phenomenon. The thing can be real and worth watching while the causal/impact claim is unsupported. Say both.
- "Unknowable denominator" is a real answer. Some rates can't be computed (the behavior evades measurement). Report that instead of inventing a number.
- Cuts both ways — check exciting claims too. FOMO hype ("everyone's 10x now") gets the same three questions as fear hype.
- Pairs with validate-the-claim (source incentives), grade-the-evidence (tiers), and disaggregate (a real aggregate impact can still hide opposite effects by group).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

