agentleFS
Sign inStart free
prosyo-aiprosyo-ai/gtm-skill/skills/outbound-ab-test/SKILL.md

outbound-ab-test

Skill13 starsChanged 22 days agoMITView on GitHub

Use this skill when you want to test what works in outbound instead of guessing: \"A/B test my cold emails\", \"design an outbound experiment\", \"which variant won\", \"testing plan for our sequences\". Picks the variable worth testing (list, angle, offer, first line, CTA, channel, timing), writes the hypothesis, splits the audience fairly, sets the sample size and decision rule, and reads the result honestly, including when it is a tie. Produces a test plan, a log entry and the decision.

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 6 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈6

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this to run outbound tests that produce decisions.

    line 16
  2. Distinctive:

    Write the stop rule: the sample size, the metric, and what counts as a win.

    line 47
  3. Distinctive:

    MUST change only one variable per test.

    line 72
  4. Distinctive:

    MUST set sample size and decision rule before launch.

    line 73
  5. Distinctive:

    NEVER declare a winner below the planned sample, and never on fewer than 200 contacts per variant.

    line 74
  6. Distinctive:

    NEVER use open rate as the deciding metric.

    line 75

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Use this to run outbound tests that produce decisions. Most outbound "tests" compare 40 sends to 40 sends and crown a winner by noise.

Inputs

  • The current campaign and its baseline numbers.
  • The metric to improve (usually positive reply rate or meetings per 100 contacts).
  • How many contacts per week the team can send to the test.

The play

1. Test the biggest lever first

Impact order, roughly: list and segment > angle and offer > first line and personalization > CTA > subject line > send time. Testing send time while the list is wrong wastes a month.

2. Write a hypothesis

"Leading with the hiring trigger instead of the pain will raise positive replies for SDR leaders, because the trigger shows why now." One variable changes; everything else stays the same.

3. Size it

Required contacts per variant depend on baseline and the lift you want to detect:

Baseline positive reply rateDetect 2x liftDetect 1.5x lift
1%~2,300 per variant~7,700 per variant
2%~1,150~3,800
4%~550~1,900

(Approximate, 80% power, 5% significance.) Rare outcomes need big samples. If volume cannot reach the size, test bigger changes (detect 2x), use total reply rate as a leading metric (it has a higher baseline), or run the test across several weeks of sends.

4. Split fairly

Random assignment within the same segment, same senders spread across both variants, same days and times. Never run variant A this week and B next week.

5. Decide in advance

Write the stop rule: the sample size, the metric, and what counts as a win. Check for guardrails too: bounce and unsubscribe rates must not rise.

6. Read and log

Report both variants with counts, the difference and whether it clears the rule. "No difference" is a result: keep the cheaper or simpler variant. Log every test so the team does not re-run old ones.

Output format

Test: <name>   Hypothesis: <...>
Variable: <one>   Held constant: <list, senders, timing, steps>
Metric: <primary> | Guardrails: bounces, opt-outs
Size: N per variant | Expected duration: N weeks
Decision rule: <...>
Readout: A x/N (x%) vs B x/N (x%) → winner / no difference → action

What good looks like

  • One variable per test.
  • Sample size decided before launch.
  • The log shows what was learned, including tests that changed nothing.
  • The common mistake is judging on open rates, which are inflated by privacy features and say nothing about replies.

Rules

  • MUST change only one variable per test.
  • MUST set sample size and decision rule before launch.
  • NEVER declare a winner below the planned sample, and never on fewer than 200 contacts per variant.
  • NEVER use open rate as the deciding metric.

Example prompts

  • "Design a test: trigger-led vs pain-led first email for VP Sales. We send 400 a week."
  • "Here are results for two variants. Is B really better?"
  • "Plan the next 3 outbound experiments for our team, in order of impact."

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free