stress-test
Skill304 starsChanged 2 months agoMITView on GitHub
Pressure-test any analysis plan or investigation design for methodological flaws before execution — wrong baselines, survivorship bias, missing segments, uncontrolled confounds, absent kill criteria. Use on `/stress-test`, "review / gut-check / stress test my analysis plan", or before committing significant time to a plan from any source. Produces a 7-point PASS/WARNING/FAIL diagnostic with an A-F grade.
19 other projects use a version of this.
Adopted by
Most recently taken up by
- tatashbar/Economic-Modeling-Studiotheir own version, first seen Nov 16, 2026
- arndvs/ctrlshfttheir own version, first seen Oct 15, 2026
- adam-paterson/skillstheir own version, first seen Oct 11, 2026
- shakacode/react_on_railstheir own version, first seen Oct 8, 2026
- sgl-project/rbgtheir own version, first seen Oct 8, 2026
- arusli1/askillstheir own version, first seen Sep 2, 2026
Used with
Projects with it also use ai-act-readiness, c-level-agents and 10 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
Hussain-ElfalahSkillai-act-readiness
Also in 3 projects that use this
Hussain-ElfalahSkillc-level-agents
Also in 3 projects that use this
MicrosoftSkillinbox-setup
Set up the Inbox agent with recommended VS Code settings and gh CLI configuration
Also in 3 projects that use this
- +9 more from Hussain-Elfalah
Other versions19 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 19
- shakacode/react_on_rails★ 5.2ksame name
- sgl-project/rbg★ 314same name
- machina-exm/film-studio-skills★ 137same name
- okikusan-public/stock_skills★ 134same name
- gbasin/stress-test-skill★ 44same name
- Hassaan146/claude-skills★ 18same name
- ilya-a11y/growthband-outbound-skills★ 13same name
- tatashbar/Economic-Modeling-Studio★ 4same name
- erhwenkuo/stock_skills★ 3same name
- Hussain-Elfalah/Agentic-Skills★ 2same name
9 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository65 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- always-compareSkill
- analysis-designSkill
- analyst-coreSkill
- archaeologySkill
- archive-analysisSkill
- auth-preflightSkill
- businessSkill
- causalSkill
- chart-to-driveSkill
- close-the-loopSkill
- codex-reviewSkill
- compare-datasetsSkill
- connect-dataSkill
- connect-snowflakeSkill
- context-compareSkill
- context-traceSkill
- data-inspectSkill
- data-mapSkill
- data-profilingSkill
- data-quality-checkSkill
- datasetsSkill
- distribution-profilerSkill
- evalSkill
- evaluate-graderSkill
- experiment-briefSkill
- experimentSkill
- exploreSkill
- exportSkill
- forecastSkill
- google-doc-exportSkill
- google-slides-exportSkill
- guardrailsSkill
- historySkill
- improve-contextSkill
- knowledge-bootstrapSkill
- log-correctionSkill
- maintain-contextSkill
- metric-specSkill
- metricsSkill
- monitor-evalsSkill
- notion-exportSkill
- paceSkill
- patternsSkill
- question-framingSkill
- question-routerSkill
- reliabilitySkill
- resume-pipelineSkill
- run-pipelineSkill
- runsSkill
- score-analysisSkill
- session-handoffSkill
- setup-dev-contextSkill
- setup-notionSkill
- setupSkill
- setup-slackSkill
- setup-snowflakeSkill
- srm-checkSkill
- stakeholder-communicationSkill
- switch-datasetSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
11 rules. 0 are common, 9 are distinctive, and 2 are very specific to this project.
0
Common
Most of the 17 edited versions we read keep these.
9
Distinctive
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 23
Before committing a week to executing an analysis plan
- Distinctive: line 24
Before presenting an analysis design to stakeholders
- Distinctive: line 25
When you've written an analysis brief and want a gut-check
- Distinctive: line 26
When reviewing someone else's analytical approach
- Distinctive: line 27
After an analysis came back with unexpected results (was the design flawed?)
- Very specific: line 45
Read the active dataset schema (.knowledge/datasets/{active}/schema.md)
- Very specific: line 46
Check if the plan's required fields/tables/dimensions exist in the dataset
- Distinctive: line 47
If ANY required data is missing → HALT, skip checkpoints 1-6, jump straight to checkpoint 7, assign FAIL verdict with BLOCKER status, …
- Distinctive: line 48
If all required data exists → proceed with checkpoints 1-7 in order
- Distinctive: line 59
After confirming data availability, review the plan against each checkpoint.
- Distinctive: line 85
Before/after with no control group
“Common” is measured: kept by at least 6 in 10 of the 17 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Skill: Stress Test
Type: Standalone — reviews any analysis plan for methodological flaws
Purpose
Takes an analysis plan, investigation design, or analytical approach and pressure-tests it for hidden flaws — wrong baselines, survivorship bias, missing segments, uncontrolled confounds, and absent kill criteria. Acts as a "senior data scientist code review" for your analytical thinking.
This skill is standalone — it works on any analysis plan, whether produced by /analysis-design, written by the user, or pulled from an existing document.
When to Use
- Before committing a week to executing an analysis plan
- Before presenting an analysis design to stakeholders
- When you've written an analysis brief and want a gut-check
- When reviewing someone else's analytical approach
- After an analysis came back with unexpected results (was the design flawed?)
Inputs
| Input | Required | Source | Description |
{{PLAN}} | Yes | User or file path | The analysis plan to review. Can be a file path, pasted text, or a description of the approach |
{{CONTEXT}} | No | User | Business context — what decision this analysis will inform |
{{AUDIENCE}} | No | User | Who will consume the results (affects what counts as "fatal" vs. "nice to have") |
{{DATA_DESCRIPTION}} | No | User | Description of available data, if not obvious from the plan |
The 7-Point Stress Test
Check data availability first. A methodologically perfect plan that needs data the dataset lacks is unexecutable, so this check precedes the seven checkpoints:
- Read the active dataset schema (
.knowledge/datasets/{active}/schema.md) - Check if the plan's required fields/tables/dimensions exist in the dataset
- If ANY required data is missing → HALT, skip checkpoints 1-6, jump straight to checkpoint 7, assign FAIL verdict with BLOCKER status, provide F grade, and stop
- If all required data exists → proceed with checkpoints 1-7 in order
Common data blockers to check for:
- Plan requires device/platform segmentation → check if device, platform, or user_agent fields exist
- Plan requires funnel analysis → check if event-level tracking exists (page views, clicks, sessions)
- Plan requires traffic source attribution → check if source, medium, campaign, or referrer fields exist
- Plan requires geographic segmentation → check if country, region, city, or geo fields exist
- Plan assumes conversion rate → check if both numerator (orders, signups) AND denominator (sessions, page views) data exist
After confirming data availability, review the plan against each checkpoint. For each one, assign a verdict: PASS, WARNING, or FAIL.
1. Hypothesis Clarity
Check: Is there a specific, testable hypothesis? Or is this a fishing expedition?
| Verdict | Criteria |
| PASS | Hypothesis is falsifiable with a clear claim about cause and effect |
| WARNING | Hypothesis exists but is vague ("engagement is down") or has multiple interpretations |
| FAIL | No hypothesis — plan is "explore the data and see what we find" |
Fix if FAIL: "State what you expect to find AND what would prove you wrong."
2. Baseline Validity
Check: Is the comparison group / baseline appropriate?
| Verdict | Criteria |
| PASS | Baseline accounts for seasonality, concurrent changes, and selection effects |
| WARNING | Baseline is reasonable but has known gaps (e.g., no YoY comparison available) |
| FAIL | Baseline is naive (e.g., "compare to last week" without checking for holidays, campaigns, or deployments) |
Common traps to check:
- Comparing to a period with a holiday, major campaign, or outage
- Before/after with no control group
- Comparing cohorts of different sizes or compositions
- Using "last month" when the metric is seasonal
Fix if FAIL: "Identify 2-3 alternative baselines and explain why each is valid or invalid."
3. Survivorship Bias
Check: Does the analysis only look at entities that "survived" to be measured?
| Verdict | Criteria |
| PASS | Analysis includes churned/dropped users, failed transactions, abandoned sessions |
| WARNING | Survivorship risk is acknowledged but not controlled for |
| FAIL | Analysis only includes "active" or "completed" entities without acknowledging what's excluded |
Common traps:
- "Average revenue per user" that only counts users who made a purchase
- "Onboarding completion rate" that only counts users who started onboarding
- "Feature satisfaction" survey that only reaches users who didn't churn
- Retention analysis that excludes users who churned before the measurement window
Fix if FAIL: "Explicitly define the denominator. Who is EXCLUDED from this analysis and why?"
4. Segment Coverage
Check: Will the analysis check for segment-level differences that could reverse the overall finding?
| Verdict | Criteria |
| PASS | Plan includes segment breakdowns on 3+ dimensions with explicit Simpson's paradox check |
| WARNING | Plan checks 1-2 segments but doesn't systematically test all categorical dimensions |
| FAIL | Plan only reports overall/aggregate numbers |
Key segments to always check:
- Device (mobile/desktop/tablet)
- Platform (iOS/Android/web)
- User tenure (new/returning/power user)
- Geography (if multi-market)
- Traffic source / acquisition channel
- Product category / feature area
- Customer tier / plan type
Fix if FAIL: "Add segment breakdowns for every categorical dimension with >2 values. Flag any reversal."
5. Confound Identification
Check: Does the plan identify and control for concurrent changes?
| Verdict | Criteria |
| PASS | Plan lists all known concurrent changes and has a control strategy for each |
| WARNING | Plan acknowledges confounds exist but doesn't have a strategy to isolate them |
| FAIL | Plan doesn't mention confounds or assumes the hypothesized cause is the only thing that changed |
Questions to surface confounds:
- What else shipped/launched/changed in the same time period?
- Were there any data pipeline or tracking changes?
- Did marketing spend or campaign strategy shift?
- Were there external events (holidays, competitor moves, news)?
- Did the user population change (growth spike, new channel, geography expansion)?
Fix if FAIL: "List every change that happened in the same time window. For each, explain how you'll separate its effect."
6. Kill Criteria
Check: Does the plan define what would make you REJECT the hypothesis?
| Verdict | Criteria |
| PASS | Explicit thresholds for acceptance AND rejection, plus practical significance criteria |
| WARNING | Has acceptance criteria but no rejection criteria (can only "confirm," never "disprove") |
| FAIL | No criteria — "we'll see what the data shows" |
What good criteria look like:
- "If the effect is less than 2pp, it's not practically significant regardless of p-value"
- "If the drop is uniform across all segments, the widget hypothesis is rejected"
- "If the confound analysis shows the loyalty program explains >60% of the variance, the widget is not the primary cause"
Fix if FAIL: "Define the minimum effect size that matters AND what finding would disprove your hypothesis."
7. Output Alignment
Check: Will the analysis output actually answer the stakeholder's question? AND can the analysis be executed with available data?
| Verdict | Criteria |
| PASS | Output format matches stakeholder need AND all required data exists |
| WARNING | Output will be useful but may need translation for the audience, OR data exists but may have quality/completeness issues |
| FAIL | Output is descriptive ("here's what happened") when stakeholder needs prescriptive ("here's what to do"), OR BLOCKER: required data does not exist (e.g., plan requires "product category" but dataset has no category field) |
Common misalignment:
- Stakeholder wants a recommendation, plan produces a description
- Stakeholder wants dollar impact, plan produces percentages
- Stakeholder wants a yes/no, plan produces "it depends"
- Stakeholder wants a 1-page brief, plan will produce a 20-page report
- BLOCKER: Plan requires data fields that don't exist in the dataset
Fix if FAIL (data blocker): "HALT — this analysis cannot be executed. The dataset does not contain [required field]. Either locate the field in another table, request a data extension, or pivot to a different dimension that exists (e.g., analyze by customer segment instead of product category)."
Fix if FAIL (alignment): "Restate the stakeholder's actual question and work backwards to what the output must contain."
Output Format
Start with a 1-sentence executive summary at the top (before the separator line) stating the grade and any blockers.
STRESS TEST RESULTS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[1-sentence summary: e.g., "GRADE D (2 FAILs, 2 WARNINGs) — significant redesign needed before execution" or "GRADE F — BLOCKER: required data does not exist"]
OVERALL GRADE: [A/B/C/D/F]
| # | Check | Verdict | Issue |
|---|-------|---------|-------|
| 1 | Hypothesis Clarity | PASS/WARNING/FAIL | ... |
| 2 | Baseline Validity | PASS/WARNING/FAIL | ... |
| 3 | Survivorship Bias | PASS/WARNING/FAIL | ... |
| 4 | Segment Coverage | PASS/WARNING/FAIL | ... |
| 5 | Confound Identification | PASS/WARNING/FAIL | ... |
| 6 | Kill Criteria | PASS/WARNING/FAIL | ... |
| 7 | Output Alignment | PASS/WARNING/FAIL | ... |
CRITICAL ISSUES (must fix before executing):
- ... [If data blocker: "BLOCKER: Cannot execute — required data does not exist. [Field name] not found in dataset."]
WARNINGS (address if time allows):
- ...
RECOMMENDED FIXES:
1. ... [state what the fix eliminates, not a time estimate]
2. ...
3. ...Grading Scale
| Grade | Criteria |
| A | 0 FAILs, 0-1 WARNINGs — execute with confidence |
| B | 0 FAILs, 2-3 WARNINGs — execute but address warnings |
| C | 1 FAIL, any WARNINGs — fix the failure before executing |
| D | 2 FAILs — significant redesign needed |
| F | 3+ FAILs — start over with the Analysis Design Brief |
Output File
Saves to: working/stress_test_{{DATE}}.md
Examples
Review a plan you wrote
/stress-test "I'm going to compare this month's conversion rate to last month's, broken by device, to see if the new checkout flow helped."Review a file
/stress-test working/investigation_plan_2026-03-28.mdWith context
/stress-test working/analysis_brief.md context="Board meeting Friday, need to decide whether to invest $500K in onboarding redesign" audience="CEO and CFO"Integration
- Works independently of
/analysis-design— use it on ANY plan - Can be chained:
/analysis-design→/stress-test→ fix → re-test - Complements the question-framing skill (which writes the Analysis Design Spec) by reviewing the plan after it's written
- Pairs with Confound Scanner agent — the scanner finds threats proactively during planning;
/stress-testreviews an existing plan retroactively
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

