design-measurement-and-experimentation
Skill7 starsChanged 4 months agoMITView on GitHub
Connect design decisions to measurable outcomes through metric trees, behavioral instrumentation, guardrails, experiment plans, and post-launch reviews. Use when a team must determine whether a design worked rather than whether it shipped.
In this repository95 other files it gives its agents.
- AGENTS.mdAGENTS.md
- accessibility-auditSkill
- agent-memory-profilesSkill
- agent-messaging-gatewaySkill
- agent-session-checkpointsSkill
- agent-skill-migrationSkill
- ai-interaction-designSkill
- animation-craftSkill
- animation-vocabularySkill
- apple-designSkill
- apple-desktop-automationSkill
- approval-sandbox-policiesSkill
- atomic-designSkill
- better-colorsSkill
- better-typographySkill
- better-uiSkill
- browser-research-agentSkill
- codex-opsSkill
- competitive-intelSkill
- component-catalogSkill
- content-designSkill
- creative-rendering-auditSkill
- cron-agent-workflowsSkill
- dashboard-from-researchSkill
- data-synthesisSkill
- design-extractSkill
- design-sandbox-proofSkill
- design-system-referenceSkill
- design-systemsSkill
- design-to-code-handoffSkill
- dialkitSkill
- docker-environmentsSkill
- emil-design-engSkill
- figma-auditSkill
- figma-code-connectSkill
- figma-create-fileSkill
- figma-design-systemsSkill
- figma-ds-rulesSkill
- figma-generate-designSkill
- figma-generate-librarySkill
- figma-implement-designSkill
- figma-library-builderSkill
- figma-plugin-apiSkill
- figma-prototypeSkill
- figma-syncSkill
- figma-useSkill
- figma-web-captureSkill
- find-animation-opportunitiesSkill
- flutter-genSkill
- gateway-ops-observabilitySkill
- hermes-agent-bridgeSkill
- hyperframes-videoSkill
- improve-animationsSkill
- information-architectureSkill
- interaction-designSkill
- interface-craft-gateSkill
- interview-researchSkill
- ios-app-intentsSkill
- ios-performance-debuggingSkill
- linear-syncSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 2 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 21
Select the evidence design: Use an experiment only when randomization is ethical and operationally valid.
- Distinctive: line 23
Validate data: Test event firing, missingness, duplicates, identity joins, timezone, bot traffic, and consistency with source-of-record …
- Very specific: line 42
Use product-discovery-and-framing to define the opportunity and usability-testing when behavioral explanation matters more than aggregate …
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Design Measurement and Experimentation
Define evidence that can change a decision. Do not manufacture certainty from weak proxies, underpowered samples, or metrics selected after results are known.
Inputs
Collect the decision, target behavior, affected population, baseline, product constraints, existing event taxonomy, data quality, expected time to effect, risk, and ability to randomize or compare.
Workflow
- State the causal hypothesis. Define the design change, expected behavior mechanism, population, outcome, and disconfirming result.
- Build a metric tree. Connect the user outcome to observable behaviors, product events, and operational measures. Separate leading indicators from lagging outcomes.
- Choose one primary metric. It must be sensitive to the intended behavior, interpretable, and difficult to improve through harmful shortcuts.
- Add guardrails. Cover accessibility, errors, time cost, trust, retention, support burden, revenue risk, and effects on excluded populations as relevant.
- Specify instrumentation. Define event name, trigger, actor, object, properties, timestamp, deduplication, privacy classification, and validation owner. Never collect data without a decision use.
- Select the evidence design. Use an experiment only when randomization is ethical and operationally valid. Otherwise choose a phased rollout, matched comparison, interrupted time series, usability benchmark, or qualitative follow-up.
- Predefine analysis. Record population, exclusions, segments, minimum detectable effect, decision threshold, duration, novelty effects, and stopping rules before reading results.
- Validate data. Test event firing, missingness, duplicates, identity joins, timezone, bot traffic, and consistency with source-of-record totals.
- Review and decide. Report effect size and uncertainty, guardrail movement, segment differences, limitations, and the decision. Preserve null and negative results.
Output contract
Return:
- causal hypothesis;
- metric tree with primary metric and guardrails;
- instrumentation specification;
- evidence or experiment design;
- pre-analysis and decision rules;
- data-quality checks;
- post-launch review date and owner.
Verification
Confirm that the primary metric reflects the user outcome, instrumentation can be tested before launch, guardrails catch plausible harm, and the planned analysis could lead to stopping or reversing the design. Explicitly state when sample size, bias, novelty, or missing data prevents a causal conclusion.
Use product-discovery-and-framing to define the opportunity and usability-testing when behavioral explanation matters more than aggregate effect.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

