experiment-design-readout
Skill3 starsChanged 8 months agoMITView on GitHub
Turn a product or growth idea into a rigorous experiment brief and post-test readout format.
In this repository19 other files it gives its agents.
- customer-discovery-jtbd-synthesisSkill
- event-tracking-planSkill
- experiment-design-readoutSkill
- launch-planSkill
- north-star-metricSkill
- onboarding-activation-auditSkill
- positioning-messagingSkill
- pricing-packaging-hypothesisSkill
- retention-diagnosisSkill
- seo-geo-content-briefSkill
- customer-discovery-jtbd-synthesisSkill
- event-tracking-planSkill
- launch-planSkill
- north-star-metricSkill
- onboarding-activation-auditSkill
- positioning-messagingSkill
- pricing-packaging-hypothesisSkill
- retention-diagnosisSkill
- seo-geo-content-briefSkill
The file
We split this skill into the rules it gives an agent
8 rules. About 0 are common, 8 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈8
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 9
Use this skill when the user wants to design, prioritize, or review product/growth experiments.
- Distinctive: line 26
Prefer minimum viable tests.
- Distinctive: line 29
Use one primary metric.
- Distinctive: line 32
Call out when a proposal is not suitable for an A/B test.
- Distinctive: line 77
Create the post-test structure before the test starts.
- Distinctive: line 143
Use this exact structure after the test finishes:
- Distinctive: line 156
If the requested test is pricing-related, state plainly when classic A/B testing is a bad fit due to low sample size, customer fairness …
- Distinctive: line 163
If the requested test has major irreversible downside, recommend a smaller-risk pretest or staged rollout.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Experiment Design Readout
Use this skill when the user wants to design, prioritize, or review product/growth experiments.
What this skill must produce
Always produce:
- Experiment brief
- Hypothesis
- Primary and secondary metrics
- Sample / ramp plan
- Implementation notes
- Decision rules
- Risks
- Post-test readout template
Working rules
- Prefer minimum viable tests.
- State the causal claim explicitly.
- Define what success and failure mean before the test launches.
- Use one primary metric.
- Guard against underpowered or low-quality tests.
- Distinguish operational guardrails from outcome metrics.
- Call out when a proposal is not suitable for an A/B test.
Step-by-step method
Step 1: define the problem
State the specific product or growth problem.
Step 2: define the hypothesis
Use this format:
If [change]
then [primary metric] will move by [expected direction / magnitude]
because [behavioral or product assumption].
Step 3: define the minimum viable test
Specify the smallest intervention that could validate or invalidate the hypothesis.
Step 4: define metrics
Choose:
- one primary metric
- 2-4 secondary metrics
- 2-4 guardrail metrics
Step 5: define the audience and segmentation
Specify:
- eligible population
- exclusions
- key segments to inspect
Step 6: define ramp and duration
Provide:
- initial ramp
- escalation path
- minimum duration
- sample size considerations
- stop conditions
Step 7: define interpretation rules
State what outcome would mean:
- launch
- iterate
- kill
- rerun with higher power / cleaner implementation
Step 8: define readout
Create the post-test structure before the test starts.
Output structure
Experiment brief
- Problem:
- User / segment:
- Proposed change:
- Why now:
Hypothesis
...
Test design
- Test type:
- Control:
- Treatment:
- Minimum viable version:
- Why this is the minimum viable test:
Metrics
Primary metric
- Name:
- Why it matters:
- Expected movement:
Secondary metrics
| Metric | Why it matters |
Guardrail metrics
| Metric | Why it matters |
Audience and segmentation
- Eligible audience:
- Exclusions:
- Segments to inspect:
Ramp / duration / sample plan
- Ramp plan:
- Target split:
- Duration:
- Sample / power caveats:
- Abort conditions:
Implementation notes
- Product / eng tasks:
- Analytics tasks:
- QA tasks:
- Dependencies:
Decision rules
- Launch if:
- Iterate if:
- Kill if:
- Rerun if:
Risks
- Underpowered test:
- Confounded implementation:
- Novelty effects:
- Long-term effects not captured:
- Maintenance cost vs gain:
Post-test readout template
Use this exact structure after the test finishes:
- Experiment summary
- What happened on the primary metric
- What happened on secondary and guardrail metrics
- Segment-level observations
- Did the result support the hypothesis?
- Most likely explanation
- Decision
- Follow-up actions
- New questions created
Special instructions
If the requested test is pricing-related, state plainly when classic A/B testing is a bad fit due to low sample size, customer fairness concerns, or poor statistical power. Offer an alternative such as:
- pricing research
- segmented rollout
- minimum viable pricing test
- sales-assisted validation
- willingness-to-pay study
If the requested test has major irreversible downside, recommend a smaller-risk pretest or staged rollout.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

