experiment
Skill16 starsChanged 16 days agoMITView on GitHub
Run every bet, growth idea or workflow change as a pre-registered experiment - hypothesis, the number that would prove it, a read date, and what result would make it wrong - written before it starts. Use for pricing, copy, onboarding, channels, cost cuts and any \"let's try\".
29 other projects use a version of this.
Adopted by
Most recently taken up by
- adithyanraj03/dsh-skills-packtheir own version, first seen Nov 15, 2026
- HiteSit/real-scientist-skillstheir own version, first seen Nov 15, 2026
- link7373/agentic-bi-teamtheir own version, first seen Nov 10, 2026
- SethGammon/Citadeltheir own version, first seen Nov 4, 2026
- DaeSeokSong/oh-my-ouroborostheir own version, first seen Nov 1, 2026
- Anglesite/anglesite-skillstheir own version, first seen Oct 27, 2026
Used with
Projects with it also use cost, eval and 9 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
maxvaegaSkillcost
Estimate what Claude Cowork / Claude Code conversations would cost if run on the pay-per-token Claude API instead of the subscription. Reports token counts and cache-aware API cost in USD and EUR, for the current conversation or for the whole history (cost per conversation). Trigger when the user asks things like "how much would this conversation cost on the API", "API cost of my Cowork conversations", "token usage / token count", "cost per conversation", "/cost", or wants to compare subscription vs API pricing.
Also in 3 projects that use this
qtzx06Skilleval
Evaluate trained YOLO model accuracy with mAP metrics and per-class error analysis. Identifies failure cases for targeted improvement. Use after training.
Also in 4 projects that use this
dirienSkilldecide
Three-agent adversarial debate protocol for strategic decisions. Two advocates steelman opposing positions while a moderator identifies risks and synthesizes a binding consensus. Use when choosing between alternatives, evaluating tradeoffs, or making high-stakes decisions.
Also in 3 projects that use this
javatarzSkillwiki
Access and manage the local GitHub Wiki clone. Use this skill when you need to read wiki content (architecture, design docs, guides), update wiki documentation, check wiki status, or push wiki changes. The wiki contains high-level system design and architectural documentation.
Also in 3 projects that use this
HeshamFSSkillloop
Run a command or check on a recurring interval
Also in 3 projects that use this
VercelSkillworkflow
Creates durable, resumable workflows using Vercel's Workflow SDK. Use when building workflows that need to survive restarts, pause for external events, retry on failure, or coordinate multi-step operations over time. Triggers on mentions of "workflow", "durable functions", "resumable", "workflow sdk", "queue", "event", "push", "subscribe", or step-based orchestration.
Also in 3 projects that use this
Other versions29 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 29
- rohanpsingh/LearningHumanoidWalking★ 1.2ksame name
- SethGammon/Citadel★ 927same name
- menkesu/awesome-pm-skills★ 441same name
- ai-analyst-lab/ai-analyst★ 304same name
- simota/agent-skills★ 99same name
- sunnnybala/Rstack★ 69same name
- Kastalien-Research/thoughtbox★ 64same name
- logly/mureo★ 64same name
- alirezarezvani/gaios★ 50same name
- skyf0xx/gambit★ 29same name
19 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository120 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- .cursor/rules/agent-agentic-workflow.mdcCursor rule
- .cursor/rules/agent-android-dev.mdcCursor rule
- .cursor/rules/agent-architect.mdcCursor rule
- .cursor/rules/agent-backend-dev.mdcCursor rule
- .cursor/rules/agent-dashboard.mdcCursor rule
- .cursor/rules/agent-debugger.mdcCursor rule
- .cursor/rules/agent-desktop-dev.mdcCursor rule
- .cursor/rules/agent-devops.mdcCursor rule
- .cursor/rules/agent-explainer-writer.mdcCursor rule
- .cursor/rules/agent-feasibility.mdcCursor rule
- .cursor/rules/agent-growth.mdcCursor rule
- .cursor/rules/agent-ios-dev.mdcCursor rule
- .cursor/rules/agent-legal.mdcCursor rule
- .cursor/rules/agent-optimizer.mdcCursor rule
- .cursor/rules/agent-orchestrator.mdcCursor rule
- .cursor/rules/agent-platform-architect.mdcCursor rule
- .cursor/rules/agent-platform-designer.mdcCursor rule
- .cursor/rules/agent-product-designer.mdcCursor rule
- .cursor/rules/agent-qa.mdcCursor rule
- .cursor/rules/agent-researcher.mdcCursor rule
- .cursor/rules/agent-reviewer.mdcCursor rule
- .cursor/rules/agent-rollback.mdcCursor rule
- .cursor/rules/agent-security.mdcCursor rule
- .cursor/rules/agent-setup.mdcCursor rule
- .cursor/rules/agent-social-media.mdcCursor rule
- .cursor/rules/agent-web-dev.mdcCursor rule
- .cursor/rules/skill-bug-fix.mdcCursor rule
- .cursor/rules/skill-commit.mdcCursor rule
- .cursor/rules/skill-context-budget.mdcCursor rule
- .cursor/rules/skill-cost-audit.mdcCursor rule
- .cursor/rules/skill-decision-log.mdcCursor rule
- .cursor/rules/skill-experiment.mdcCursor rule
- .cursor/rules/skill-founder-voice.mdcCursor rule
- .cursor/rules/skill-frontend-design.mdcCursor rule
- .cursor/rules/skill-gates.mdcCursor rule
- .cursor/rules/skill-grilling.mdcCursor rule
- .cursor/rules/skill-handoff.mdcCursor rule
- .cursor/rules/skill-human-design.mdcCursor rule
- .cursor/rules/skill-human-steps.mdcCursor rule
- .cursor/rules/skill-karpathy-guidelines.mdcCursor rule
- .cursor/rules/skill-known-reds.mdcCursor rule
- .cursor/rules/skill-legal-review.mdcCursor rule
- .cursor/rules/skill-new-feature.mdcCursor rule
- .cursor/rules/skill-observability.mdcCursor rule
- .cursor/rules/skill-ordered-steps.mdcCursor rule
- .cursor/rules/skill-performance.mdcCursor rule
- .cursor/rules/skill-readiness-gate.mdcCursor rule
- .cursor/rules/skill-release.mdcCursor rule
- .cursor/rules/skill-requirement-first.mdcCursor rule
- .cursor/rules/skill-safe-data-write.mdcCursor rule
- .cursor/rules/skill-setup.mdcCursor rule
- .cursor/rules/skill-single-source.mdcCursor rule
- .cursor/rules/skill-spec-audit.mdcCursor rule
- .cursor/rules/skill-tailoring.mdcCursor rule
- .cursor/rules/skill-twin-check.mdcCursor rule
- .cursor/rules/skill-verified-claims.mdcCursor rule
- .cursor/rules/skill-verify-green.mdcCursor rule
- bug-fixSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
3 rules. 0 are common, 1 is distinctive, and 2 are very specific to this project.
0
Common
Most of the 27 edited versions we read keep these.
1
Distinctive
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 10
Write the entry in docs/EXPERIMENTS.md before the change ships.
- Distinctive: line 38
When the change is something users see (copy, onboarding, a price page, a feature) and enough of them arrive, test it on a random sample …
- Very specific: line 52
No peeking: Read once, at the planned size or the read date.
“Common” is measured: kept by at least 6 in 10 of the 27 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
<!-- Generated by tools/sync_adapters.py from skills/experiment/SKILL.md. Edit the source, not this file. -->Experiment
Write the entry in docs/EXPERIMENTS.md before the change ships. An experiment written after
the result is a story.
The entry
### 7. Annual plan at two months free, from 2026-03-01
- **Why:** monthly churn is 9 percent; annual buyers churn less.
- **Change:** one change only: the annual option on the pricing page.
- **Hypothesis:** some monthly buyers choose annual.
- **Prediction:** at least 15 percent of new paid sign-ups pick annual within 30 days.
- **Wrong if:** under 5 percent pick annual. Then the annual option comes off and we look at
churn reasons instead.
- **Confounds:** the launch post on 2026-03-03 brings unusual traffic.
- **Read:** 2026-03-31.And one line in the "At a glance" table at the top of the file.
Rules
- One change at a time, or the result cannot be attributed.
- The read date is kept. Reading early because it looks good is how false wins happen.
- Record what the numbers say, not what was hoped. "Not distinguishable from noise" is a result.
- A closed experiment moves to
docs/EXPERIMENTS_CLOSED.mdwith its numbers and verdict.
A/B tests: when users can be split
When the change is something users see (copy, onboarding, a price page, a feature) and enough of them arrive, test it on a random sample instead of before and after: half see the current version (A), half the new one (B), at the same time, so a news day or a launch post hits both.
- Size first. Pick one primary metric, today's rate and the smallest change worth acting
on, then run
python tools/ab_size.py size --base <rate> --target <rate>. If that many users will not arrive before the read date, do not run an A/B test: run it before and after, and say the result is weaker. - Assign at random, once. A user's arm comes from a stable hash of their user id and the experiment's name, is logged when assigned, and never changes. Both arms ship behind the flag (TPL-73), so B is switched off in one step.
- Guardrails. Name the numbers that must not get worse (errors, refunds, unsubscribes, AI spend per user) and the drop that stops the test early. A guardrail is the only reason to stop before the planned size.
- No peeking. Read once, at the planned size or the read date. First run
python tools/ab_size.py srm --a <users> --b <users>: a mismatch means assignment is broken and the result is void. Thenreadfor the difference and its p-value. - Ask first. Showing users different prices or terms goes to
legaland the founder before it starts; the dashboard shows each arm's count and metrics from one place (skills/single-source/).
An A/B entry adds four lines to the template above, and tools/checks/experiments_check.py
refuses one without them:
- **Arms:** A, the current welcome screen; B, the three-step welcome screen. 50/50.
- **Assignment:** hash of user id and "welcome-v2", logged at first open.
- **Sample:** 3,839 per arm (`tools/ab_size.py size --base 0.10 --target 0.12`).
- **Guardrails:** crash rate; stop if B's is double A's.Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

