agentleFS
Sign inStart free
ThinkingAIAgenticEngineThinkingAIAgenticEngine/scenario-skills/skills/analysis-guides/ab-test-analysis/SKILL.md

ab-test-analysis

Skill6 starsChanged 29 days agoNo licenceView on GitHub

A/B test significance analysis for two-group experiments. Before querying it aligns variant-field aliases, validates field value domains and enforces an exposure-cohort caliber (success users must be a subset of the exposed cohort and occur after exposure), then runs two-proportion z-tests and Welch's t-tests with confidence intervals, Cohen's d, MDE, a Bonferroni-corrected decision rule over the declared metric family, Bayesian posterior, post-hoc power, business significance and polarity-aware guardrail checks. Every verdict, recommendation and guardrail conclusion reads the corrected decision flag, never the raw p-value. Emits a self-contained Chinese-language HTML report with verdict, per-metric recommendations and an action plan; a five-metric experiment renders in about 3-5 minutes instead of 1-2 hours of manual spreadsheet work. Use for A/B test review, variant win check, lift and significance questions. Do NOT use for experiment design, traffic allocation or pre-launch sample-size planning, experiment configuration management, tracking-plan authoring or event instrumentation, tests with more than five arms, or when no variant property exists in tracking.

15 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use brainstorm-experiments-new, brainstorm-experiments-existing and 10 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • skytiger6724Skill

    brainstorm-experiments-new

    Design lean startup experiments (pretotypes) for a new product. Creates XYZ hypotheses and suggests low-effort validation methods like landing pages, explainer videos, and pre-orders. Use when validating a new product idea, creating pretotypes, or testing market demand.

    Also in 7 projects that use this

  • skytiger6724Skill

    brainstorm-experiments-existing

    Design experiments to test assumptions for an existing product — prototypes, A/B tests, spikes, and other low-effort validation methods. Use when validating assumptions, testing feature ideas cheaply, or planning product experiments.

    Also in 5 projects that use this

  • 45ckSkill

    pestle-analysis

    Also in 5 projects that use this

  • launchapp-devSkill

    interview-script

    Create a structured user interview script with warm-up, core exploration, and wrap-up sections. Use when preparing for user research interviews to ensure consistent, insightful conversations.

    Also in 7 projects that use this

  • AnthropicSkill

    sql-queries

    Write correct, performant SQL across all major data warehouse dialects (Snowflake, BigQuery, Databricks, PostgreSQL, etc.). Use when writing queries, optimizing slow SQL, translating between dialects, or building complex analytical queries with CTEs, window functions, or aggregations.

    Also in 7 projects that use this

  • nandanosqlSkill

    swot-analysis

    Conduct a structured SWOT analysis — Strengths, Weaknesses, Opportunities, Threats — that connects internal capabilities to external forces for strategic decision-making.

    Also in 8 projects that use this

  • +6 more from skytiger6724

The file

Show its rules and text

We split this skill into the rules it gives an agent

28 rules. 0 are common, 12 are distinctive, and 16 are very specific to this project.

  • 0

    Common

    Most of the 12 edited versions we read keep these.

  • 12

    Distinctive

    This author's own choices. Worth a look.

  • 16

    Very specific

    Tied to their project's names, files or tools. Probably skip.

No licence file, so all rights are reserved — read it at the source. We count its rules but don't quote them. Read them on GitHub.

“Common” is measured: kept by at least 6 in 10 of the 12 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free