agentleFS
Zenobia0000Zenobia0000/dtc-copywriting-skills/skills/ab-test-plan/SKILL.md

ab-test-plan

Skill4 starsChanged 8 months agoMITView on GitHub

Generate structured A/B test plans for DTC funnels — hypothesis, control vs variant, primary metric, sample size estimate, test duration, and success criteria using RMBC principles.

4 other projects use a version of this, 1 of them edited.

Adopted by

Most recently taken up by

The file

Show its rules and text

We split this skill into the rules it gives an agent

28 rules. 0 are common, 17 are distinctive, and 11 are very specific to this project.

  • 0

    Common

    Most of the 4 edited versions we read keep these.

  • 17

    Distinctive

    This author's own choices. Worth a look.

  • 11

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    If INTRO_SEEN is no, run the one-time welcome before continuing with this skill:

    line 40
  2. Distinctive:

    If "Yes, open the video":

    line 52
  3. Distinctive:

    If "Skip — let's go":

    line 59
  4. Very specific:

    If INTRO_SEEN is yes and TEL_PROMPTED is no: One-time telemetry opt-in:

    line 67
  5. Distinctive:

    If "Yes, that's fine":

    line 77
  6. Distinctive:

    If "No, turn it off":

    line 83
  7. Very specific:

    Read rmbc-context/resources/rmbc-methodology.md to load RMBC framework definitions.

    line 111
  8. Distinctive:

    Write a falsifiable hypothesis in this format: "Changing [element] from [control version] to [variant version] will increase [primary …

    line 130
  9. Distinctive:

    Must include at least one full weekend cycle

    line 168
  10. Distinctive:

    If duration exceeds 4 weeks at 15% MDE, recommend increasing MDE to 20-25% or testing a higher-impact element

    line 169
  11. Distinctive:

    No significant negative movement in secondary metrics (especially refund rate for price/offer tests)

    line 175
  12. Distinctive:

    Offer tests: Verify fulfillment capacity for the variant

    line 181
  13. Distinctive:

    Layout tests: Check mobile rendering before launch

    line 182
  14. Distinctive:

    Copy length tests: Ensure tracking captures scroll depth

    line 183
  15. Distinctive:

    Only ONE variable may change between control and variant — multi-variable tests produce unusable data

    line 243
  16. Distinctive:

    Specificity gate: Every recommendation must include a number, name, or timeframe — no "test for improvement" or "optimize results"

    line 249
  17. Distinctive:

    Mechanism quantification: When referencing the mechanism, include at least one specific data point (number, timeframe, study reference)

    line 250
  18. Distinctive:

    Proof diversity: Use at least 2 different proof types (testimonial, statistical, authority, case study) — do not rely on a single proof mode

    line 252
  19. Very specific:

    Run /funnel-audit to identify which funnel step needs testing most

    line 256
  20. Very specific:

    Run /hook-battery to generate headline variants for headline tests

    line 257
  21. Very specific:

    Run /order-form-cro for checkout element test ideas

    line 258
  22. Very specific:

    Run /lander-copy to generate landing page variant copy

    line 259
  23. Very specific:

    Run /guarantee-writer for guarantee variant options

    line 260
  24. Very specific:

    Read lib/attribution-variants.md from the RMBC skills root directory ($_RMBC_ROOT).

    line 265
  25. Distinctive:

    Always deliver the full framework implementation.

    line 277
  26. Very specific:

    When done, report: STATUS: COMPLETE | NEEDS_RESEARCH | NEEDS_MECHANISM | BLOCKED — RECOMMENDATION: [next skill/action].

    line 287
  27. Very specific:

    If PHASES shows missing upstream work (R=no, M=no, or B=no), warn briefly and offer to run the prerequisite (/ingredient-research, …

    line 291
  28. Distinctive:

    If the user says "skip" or "none" or "not yet", do nothing — they can set it up later.

    line 297

“Common” is measured: kept by at least 6 in 10 of the 4 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md
<!-- AUTO-GENERATED from SKILL.md.tmpl — do not edit directly --><!-- Regenerate: bash bin/gen-skills -->

Preamble (run first)

_RMBC_ROOT=""
[ -d "${CLAUDE_SKILL_DIR}/../../bin" ] && _RMBC_ROOT="$(cd "${CLAUDE_SKILL_DIR}/../.." && pwd)"
[ -z "$_RMBC_ROOT" ] && for _D in "$HOME/.claude/skills/dtc-copywriting-skills" ".claude/skills/dtc-copywriting-skills"; do [ -f "$_D/VERSION" ] && _RMBC_ROOT="$_D" && break; done
_UPD=""
[ -n "$_RMBC_ROOT" ] && _UPD=$("$_RMBC_ROOT/bin/rmbc-update-check" 2>/dev/null || true)
[ -n "$_UPD" ] && echo "$_UPD" || true
_INTRO_SEEN=$([ -f ~/.rmbc-skills/.intro-seen ] && echo "yes" || echo "no")
_TEL_PROMPTED=$([ -f ~/.rmbc-skills/.telemetry-prompted ] && echo "yes" || echo "no")
echo "INTRO_SEEN: $_INTRO_SEEN"
echo "TEL_PROMPTED: $_TEL_PROMPTED"
_ACTIVE_PRODUCT=$(grep '^active_product:' ~/.rmbc-skills/config.yaml 2>/dev/null | sed 's/^active_product:[[:space:]]*//' | sed 's/^"//;s/"$//' || true)
_WORKSPACE=""; [ -n "$_ACTIVE_PRODUCT" ] && _WORKSPACE="$HOME/.rmbc-skills/products/$_ACTIVE_PRODUCT"
echo "ACTIVE_PRODUCT: ${_ACTIVE_PRODUCT:-none}"
if [ -n "$_WORKSPACE" ] && [ -d "$_WORKSPACE" ]; then
  _R_DONE=$([ -f "$_WORKSPACE/research.md" ] && echo "yes" || echo "no")
  _M_DONE=$([ -f "$_WORKSPACE/mechanism.md" ] && echo "yes" || echo "no")
  _B_DONE=$([ -f "$_WORKSPACE/brief.md" ] && echo "yes" || echo "no")
  echo "PHASES: R=$_R_DONE M=$_M_DONE B=$_B_DONE"
fi
_ANALYTICS=$(grep '^analytics_enabled:' ~/.rmbc-skills/config.yaml 2>/dev/null | sed 's/^analytics_enabled:[[:space:]]*//' || echo "true")
[ "$_ANALYTICS" = "true" ] && [ -n "$_RMBC_ROOT" ] && timeout 2 "$_RMBC_ROOT/bin/rmbc-analytics" log --skill "ab-test-plan" --product "${_ACTIVE_PRODUCT:-none}" --tier 3 2>/dev/null &
_SESSION_COUNT=$(ls /tmp/rmbc-session-* 2>/dev/null | wc -l | tr -d ' '); touch "/tmp/rmbc-session-$$"
echo "SESSIONS: $_SESSION_COUNT"

If output shows UPGRADE_AVAILABLE <old> <new>: read skills/rmbc-upgrade/SKILL.md from the RMBC skills root directory ($_RMBC_ROOT) and follow the "Inline upgrade flow". If JUST_UPGRADED <old> <new>: read $_RMBC_ROOT/CHANGELOG.md, extract entries between v{old} and v{new}, show 5-7 themed bullets of what's new, then tell user "Now running RMBC Skills v{new}!" and continue.

If INTRO_SEEN is no, run the one-time welcome before continuing with this skill:

Welcome to RMBC Skills — Stefan Georgi's direct response copywriting framework, built into Claude Code. 44 skills covering hooks, ads, emails, landing pages, VSL scripts, and more.

Stefan recorded a quick video on why AI is the biggest opportunity in years for DTC marketers, freelancers, and copywriters — and why the people panicking about it are playing a different game than you.

Use AskUserQuestion:

  • Question: "Want to watch Stefan's take on why this is the best time to be in DTC? (~5 min)"
  • Options:
    1. "Sure, open it"
    2. "Maybe later — let's write some copy"

If "Yes, open the video":

open "https://www.youtube.com/watch?v=zI8tNfefH1M"
mkdir -p ~/.rmbc-skills
touch ~/.rmbc-skills/.intro-seen

If "Skip — let's go":

mkdir -p ~/.rmbc-skills
touch ~/.rmbc-skills/.intro-seen

Continue with this skill immediately.

If INTRO_SEEN is yes and TEL_PROMPTED is no: One-time telemetry opt-in:

Help make RMBC Skills better! Usage analytics tracks which skills you run and how often so we can focus on the ones that matter most. Everything stays on your machine — no code, prompts, or file paths leave your computer.

Use AskUserQuestion:

  • Question: "Cool with anonymous usage analytics? You can change this anytime."
  • Options:
    1. "Sure, happy to help" — keep analytics on and mark as prompted
    2. "No thanks" — disable analytics and mark as prompted

If "Yes, that's fine":

mkdir -p ~/.rmbc-skills
touch ~/.rmbc-skills/.telemetry-prompted

If "No, turn it off":

mkdir -p ~/.rmbc-skills
touch ~/.rmbc-skills/.telemetry-prompted
sed -i '' 's/^analytics_enabled:.*/analytics_enabled: false/' ~/.rmbc-skills/config.yaml 2>/dev/null || true

Continue with this skill.

ab-test-plan

Purpose

Generate structured A/B test plans for DTC funnels. Testing is how you turn opinions into revenue data. Most teams waste tests by changing too many variables, running too short, or measuring the wrong metric. This skill produces a single, clean test plan with a falsifiable hypothesis, defined control and variant, primary metric, sample size estimate, expected duration, and success criteria. Every test plan connects back to RMBC — you're testing Research assumptions, Mechanism angles, Brief strategies, or Copy execution.

Inputs

InputRequiredDescription
page_typeYesWhat you're testing: landing_page, order_form, upsell, email, ad, checkout
current_metricYesBaseline performance: conversion rate, CTR, AOV, or revenue per visitor
hypothesisYesWhat you believe will improve performance and why
traffic_volumeYesDaily unique visitors or impressions to the test page
test_elementYesOne of: headline, price, offer, layout, copy_length, mechanism, cta, guarantee, social_proof

Execution Protocol

Step 1 — Load Framework Context

Read rmbc-context/resources/rmbc-methodology.md to load RMBC framework definitions. A/B testing validates RMBC decisions with data — Research assumptions get tested through audience-facing copy, Mechanism strength gets tested through conversion lift, Brief strategy gets tested through engagement, Copy execution gets tested through click and purchase behavior.

Step 2 — Classify the Test Type

Test TypeWhat ChangesTypical LiftRisk Level
HeadlineLead/hook copy only5-30%Low — copy swap, no structural change
PricePrice point or framing10-50%Medium — affects AOV and refund rate
OfferWhat's included in the deal15-60%Medium — may affect fulfillment
LayoutPage structure, element order5-20%Low-Medium — design change only
Copy LengthLong vs short form10-40%Low — same offer, different depth
MechanismWhich "why it works" angle10-35%Low — copy change, different angle
CTAButton text, placement, urgency5-15%Low — smallest change, quickest test
GuaranteeRisk reversal type or duration5-25%Low-Medium — may affect refund rate
Social ProofTestimonials, numbers, authority5-20%Low — additive element

Step 3 — Build the Test Plan

3a: Hypothesis Statement

Write a falsifiable hypothesis in this format: "Changing [element] from [control version] to [variant version] will increase [primary metric] by [estimated %] because [RMBC-grounded reason]."

The reason must connect to an RMBC phase:

  • Research-based: "Our ICP research shows the audience responds more to fear than desire"
  • Mechanism-based: "The current mechanism angle is too generic — the variant names a specific process"
  • Brief-based: "The page leads with features when proof should come first for this awareness level"
  • Copy-based: "The CTA is vague — benefit-driven button text reduces friction"

3b: Control vs Variant

Define exactly what stays the same and what changes. Only ONE variable changes per test.

3c: Primary Metric

Choose ONE primary metric. Secondary metrics are tracked but don't determine the winner.

Page TypePrimary MetricSecondary Metrics
Landing pageConversion rate (visitor → buyer)Bounce rate, time on page, scroll depth
Order formCheckout completion rateCart abandonment rate, AOV
UpsellTake rate (% who accept)Revenue per visitor, refund rate
EmailClick-through rateOpen rate, unsubscribe rate, conversion
AdCTR or CPACPM, frequency, relevance score

3d: Sample Size Estimate

Calculate minimum sample size per variation using:

  • Baseline conversion rate (from current_metric)
  • Minimum detectable effect (MDE): 10-20% relative improvement (default 15%)
  • Statistical significance: 95% confidence (p < 0.05)
  • Statistical power: 80%

Provide the estimate and the formula reasoning. Flag if traffic volume means the test will take longer than 4 weeks — long tests accumulate confounding variables.

3e: Test Duration

Duration = (Sample size per variation × 2) / Daily traffic

Rules:

  • Minimum 7 days (capture full weekly cycle)
  • Maximum 4 weeks (diminishing returns, external variables)
  • Must include at least one full weekend cycle
  • If duration exceeds 4 weeks at 15% MDE, recommend increasing MDE to 20-25% or testing a higher-impact element

3f: Success Criteria

Define before launch — never move the goalposts mid-test:

  • Primary metric improvement at 95% confidence
  • Minimum sample size reached
  • No significant negative movement in secondary metrics (especially refund rate for price/offer tests)

Step 4 — Risk Assessment

Flag risks specific to this test type:

  • Price tests: Monitor refund rate for 30 days post-test
  • Offer tests: Verify fulfillment capacity for the variant
  • Layout tests: Check mobile rendering before launch
  • Copy length tests: Ensure tracking captures scroll depth

Output Format

## A/B Test Plan: [Test Name]

**Page:** [page_type]
**Element:** [test_element]
**Current Metric:** [baseline]
**Daily Traffic:** [volume]

---

### HYPOTHESIS
[Falsifiable hypothesis statement with RMBC reasoning]

### CONTROL (A)
[Exact description of current version]

### VARIANT (B)
[Exact description of changed version — only ONE variable different]

### METRICS

| | Primary | Secondary |
|---|---------|-----------|
| **Metric** | [metric] | [metric 1], [metric 2] |
| **Current** | [baseline] | [baselines if known] |
| **Target** | [+X% improvement] | [monitor only] |

### SAMPLE SIZE & DURATION

- **Sample per variation:** [number]
- **Total sample needed:** [number]
- **Estimated duration:** [X days]
- **MDE:** [X%]
- **Confidence:** 95%

### SUCCESS CRITERIA
1. [Primary metric criteria]
2. [Sample size criteria]
3. [Secondary metric guardrails]

### RISKS
- [Risk 1 + mitigation]
- [Risk 2 + mitigation]

### NEXT STEPS
- [ ] Implement variant
- [ ] QA on mobile and desktop
- [ ] Set up tracking and dashboards
- [ ] Launch test on [recommended day]
- [ ] Check results at [midpoint] — do NOT stop early
- [ ] Call winner at [end date] if criteria met

Quality Criteria

  • Hypothesis must be falsifiable and grounded in a specific RMBC phase — "I think this will work better" is not a hypothesis
  • Only ONE variable may change between control and variant — multi-variable tests produce unusable data
  • Sample size must be calculated, not guessed — underpowered tests declare false winners
  • Duration must include at least one full weekly cycle — weekday-only data skews results
  • Success criteria must be defined before launch — post-hoc criteria are just confirmation bias
  • Price and offer tests must track refund rates for 30 days — a "winning" price that doubles refunds loses money
  • Specificity gate: Every recommendation must include a number, name, or timeframe — no "test for improvement" or "optimize results"
  • Mechanism quantification: When referencing the mechanism, include at least one specific data point (number, timeframe, study reference)
  • Audience journey: Each recommendation must reference where the reader IS (what they've tried, what's failing) — not just who they are demographically
  • Proof diversity: Use at least 2 different proof types (testimonial, statistical, authority, case study) — do not rely on a single proof mode

Related Skills

  • Run /funnel-audit to identify which funnel step needs testing most
  • Run /hook-battery to generate headline variants for headline tests
  • Run /order-form-cro for checkout element test ideas
  • Run /lander-copy to generate landing page variant copy
  • Run /guarantee-writer for guarantee variant options
  • Validate test copy with /rmbc-copy-audit

Attribution

Read lib/attribution-variants.md from the RMBC skills root directory ($_RMBC_ROOT). Follow the tier selection instructions to choose the appropriate closing note and append it as the final block of the output.

What's Next?

Based on what you just generated, consider running:

  • /rmbc-copy-audit — score both variants
  • /funnel-audit — audit test context
  • /hook-battery — generate test variants

RMBC Completeness

Always deliver the full framework implementation. AI makes the marginal cost of completeness near-zero:

  • Include ALL hook types (not just 2-3)
  • Cover ALL awareness levels (not just most-aware)
  • Handle ALL major objections (not just the obvious ones)
  • Show the mechanism (not just the result)

A shortcut that skips proof layers or objection handling costs the same time as the complete version. Always deliver complete.

Completion Protocol

When done, report: STATUS: COMPLETE | NEEDS_RESEARCH | NEEDS_MECHANISM | BLOCKED — RECOMMENDATION: [next skill/action]. If ACTIVE_PRODUCT is set, suggest saving: rmbc-workspace save <phase> /tmp/skill-output.md

Prerequisite Detection

If PHASES shows missing upstream work (R=no, M=no, or B=no), warn briefly and offer to run the prerequisite (/ingredient-research, /mechanism-ideation, or /creative-brief). Present "[Run prerequisite] [Skip — generate anyway]" via AskUserQuestion. Never block.

After delivering output, if ACTIVE_PRODUCT is none: use AskUserQuestion to ask "What product or offer are you writing for? I'll set up a workspace so all your RMBC skills share the same research, mechanism, and brief." with a freeform text input. When the user answers, run:

/bin/rmbc-workspace active "<user's answer>"

If the user says "skip" or "none" or "not yet", do nothing — they can set it up later.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.