bug-reproduction
Skill2 starsChanged 7 days agoApache-2.0View on GitHub
Turn a vague bug report into a verified minimal reproduction and a failing regression test: implicit repro extraction (env, build, steps, data), reproduce-minimize-isolate-capture, git bisect, deterministic repro, red test before the fix. Use when: reproduce this bug, minimal reproduction, repro steps, git bisect, regression test for a defect, can't reproduce this bug. Not for failure triage without reproducing, or spec-based test generation.
5 other projects use a version of this, 2 of them edited.
Adopted by
Most recently taken up by
- mlschodkowski/skillstheir own version, first seen Nov 16, 2026
- jason0971/skillsedited, first seen Nov 8, 2026
- petrkindlmann/qa-skillsedited, first seen Oct 25, 2026
- DodoraApp/DodoStreamtheir own version, first seen Oct 8, 2026
- goca-se/dotcontexttheir own version, first seen Jul 7, 2026
Other versions5 projects use a version of this.
Word for word · 0
None found.
Edited · 2
- petrkindlmann/qa-skills★ 27468% same
- jason0971/skills★ 170% same
Wrote their own version · 3
- DodoraApp/DodoStream★ 248same name
- goca-se/dotcontext★ 9same name
- mlschodkowski/skills★ 1same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository34 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- accessibility-testingSkill
- agentswitchSkill
- api-testingSkill
- bug-reportSkill
- demoSkill
- deploy-to-stageSkill
- exploratory-testingSkill
- failure-analysisSkill
- model-setupSkill
- onboard-productSkill
- onboard-projectSkill
- perf-analysisSkill
- qa-helpSkill
- qa-metricsSkill
- qa-reportSkill
- re-boardSkill
- reconSkill
- release-readinessSkill
- report-to-jiraSkill
- retest-bugSkill
- retro-learnSkill
- risk-based-testingSkill
- scaffold-suiteSkill
- security-testingSkill
- skill-reviewSkill
- test-automationSkill
- test-case-managementSkill
- test-reliabilitySkill
- test-ticketSkill
- visual-testingSkill
- webdriverio-testingSkill
- whats-newSkill
The file
We split this skill into the rules it gives an agent
9 rules. 0 are common, 7 are distinctive, and 2 are very specific to this project.
0
Common
Most of the 5 edited versions we read keep these.
7
Distinctive
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 109
Write these into a single repro spec before touching code.
- Distinctive: line 124
MINIMIZE: Remove one step, field, or dependency.
- Very specific: line 142
Use git bisect with an automated test script (exit 0 = good, exit 1 = bad, exit 125 = skip).
- Distinctive: line 189
Run it and confirm it fails before the fix — it must be red first.
- Distinctive: line 192
After the dev's fix lands, re-run: it should flip to green.
- Distinctive: line 219
Only when the test is red-without-fix and green-with-fix, repeatably, is the fix verified.
- Very specific: line 266
Avoid: fixing before reproducing, minimizing before confirming, removing multiple variables at once, manual bisect, exit 1 on build failure …
- Distinctive: line 303
If the bug is a regression, git bisect has named the introducing commit SHA and it is recorded in the ticket.
- Distinctive: line 312
If it did not reproduce, it is classified (flaky / environment-specific / data-dependent / not-reproducible) with the evidence that led …
“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
<objective>
A bug you cannot reproduce is a bug you cannot fix or prove fixed. This skill takes a
thin, hand-wavy report ("order total is wrong sometimes") and drives it to a VERIFIED
minimal reproduction, a deterministic failing test written BEFORE the fix, a `git bisect`
that names the introducing commit, and a structured evidence block in the ticket. The
discipline it enforces: reproduce before theorizing, minimize one cut at a time, freeze
time/seed/network so the repro fails identically every run, watch the test go red first,
and confirm the fix flips it green — and that reverting the fix turns it red again.
</objective>Quick Route
| You have… | Go to |
| A thin report and no idea how to trigger it | Step 1: Extract the implicit repro |
| A messy 14-step repro to clean up | Step 2: The reproduce-minimize-isolate-capture loop |
| "Worked last month, broken now" | Step 3: Bisect to the introducing commit |
| A repro that passes/fails inconsistently | Step 4: Make the repro deterministic |
| A clean repro, no fix yet | Step 5: Write the failing regression test (red) first |
| "The dev says it's fixed, test is green" | Step 6: Verify the fix actually fixes it |
| "It won't reproduce for me but does for the user" | Step 7: Flaky vs environment vs not-reproducible |
| Repro + test done | Step 8: Write the evidence back into the ticket |
Discovery Questions
First, check the project bridge file .agents/qa-project-context/<KEY>.md — it carries the tech
stack, test runner, known-flaky areas, and environment matrix. Pass over any question it
already answers. If it is missing, suggest /onboard-project (or /re-board) — the bridge
file is generated there, never hand-written.
- What is the report, verbatim? The thinner it is, the more you must extract before touching code — a one-line report sets the whole intake agenda (Step 1).
- Does it reproduce at all yet, and how reliably? "Every time" vs "sometimes" decides whether you go straight to minimizing or into determinism work first.
- Did it ever work? A known-good past release unlocks
git bisectto the introducing commit; no known-good point means you debug forward instead. - What is the test runner and stack? Vitest/Jest vs Playwright changes the determinism
API (
vi.setSystemTimevspage.clock) and where the regression test lands. - What are the non-determinism sources? Time-of-day rules, randomness, third-party APIs, locale — each must be pinned for the repro to be trustworthy.
Core Principles
- Reproduce before you theorize. The strongest wrong instinct is to read a symptom and jump to a root cause ("sounds like a float rounding bug, let me patch the total calc"). Don't. Extract the repro, make it fail on demand, and only then form a hypothesis. A fix without a reproduction is a guess you can't falsify.
- A repro is a deterministic artifact, not a story. "It happens around midnight with a random code" is a story. Freeze the clock, seed the RNG, and stub the network so the same inputs produce the same failure on every run and every machine. If it isn't deterministic, you can't bisect it, test it, or prove it fixed.
- Minimize one variable at a time, and re-confirm after every cut. Shrinking the repro is a search, not a rewrite. Remove one step, data field, or dependency, then re-run and confirm it still reproduces. Removing several at once tells you nothing about which one mattered.
- The regression test is written RED, before the fix. Assert the real expected value, watch it fail first (proving it catches this bug), then watch the fix flip it green. A test added after the fix, or one disabled / marked pending, proves nothing.
- Green isn't done — revert-to-verify is. A passing test can pass for the wrong reason. Temporarily remove the fix and confirm the test goes red again. Only a test that fails without the fix actually guards against the bug.
Step 1: Extract the implicit repro
A thin report ("Checkout is broken, order total is wrong sometimes") names a symptom, not a path. Before any code, extract or ask for every reproducibility dimension. Never invent the repro steps from imagination, and never theorize a root cause yet — both come after the bug reproduces.
For a "wrong total" / data-correctness bug, the load-bearing dimensions a thin report most often omits are:
- Exact steps to reproduce — the click-by-click path, not "checkout is broken."
- Build / version / commit (git SHA) — they may be on a build where it's already fixed.
- Environment — browser + version, OS, device.
- Input data — cart contents, quantities, the account/user, coupon, the exact fixture. A total is a pure function of its inputs; without them you are guessing.
- Expected vs actual — the number they expected and the number they saw.
- Frequency — every time, or intermittent? "Sometimes" points at non-determinism.
- Locale / timezone / currency — rounding, tax, and formatting are locale-specific; a
total "wrong" in
de-DEmay be correct inen-US. - Timestamp of occurrence, plus any logs/screenshots/network trace.
Write these into a single repro spec before touching code. If a row is blank, that is your next question to the reporter — not a license to start writing the fix or theorizing.
See references/intake.md for the full extraction checklist, why each load-bearing
dimension matters for "wrong total," and the repro-spec template.
Step 2: The reproduce→minimize→isolate→capture loop
Given a confirmed-but-messy repro (e.g. 14 manual UI steps across 3 pages), do not hand the 14-step version to the developer and do not rewrite it wholesale. Run this loop:
- REPRODUCE / confirm. First establish a baseline: run the full repro and confirm it actually fails. You can only minimize something that currently reproduces.
- MINIMIZE. Remove one step, field, or dependency. Re-run. If it still reproduces, keep the cut; if it no longer fails, that element was load-bearing — restore it. Repeat, one variable at a time, until every remaining piece is necessary. This is the core ordering rule: never minimize before confirming it reproduces, and never omit verifying it still fails after each cut.
- ISOLATE. Narrow the failure to the smallest layer that still shows it — drop from a 3-page UI flow to a single page, then to a unit/API call against the offending function if the bug lives below the UI.
- CAPTURE. Record the now-minimal repro as evidence: the smallest steps or the single command, plus logs/trace/screenshot. This is what the developer and the regression test consume.
The output is the smallest sequence that still reproduces — not the original walkthrough.
Step 3: Bisect to the introducing commit
Use git bisect with an automated test script (exit 0 = good, exit 1 = bad,
exit 125 = skip). Run the reproduce script at each step. Handle flaky commits
with bisect skip. Never bisect manually or by eyeballing the suite.
Full bisect commands, skip handling, and determinism workarounds:
references/bisect-and-anti-patterns.md → "Bisect".
Step 4: Make the repro deterministic
The bug "only around midnight, with a random discount code, via a third-party pricing API"
has three non-determinism sources. Pin all three so it fails the same way every single
run — do not wait for midnight, do not let it hit the real pricing API, and never use a
sleep/setTimeout/waitForTimeout to paper over timing.
| Source | Vitest | Playwright |
| Time | vi.useFakeTimers() + vi.setSystemTime(new Date('…')) | page.clock.install({ time }) + page.clock.setFixedTime(…) |
| Randomness | faker.seed(1337) (or stub Math.random) | seed the app's RNG via an init hook |
| Network | MSW setupServer + http.get → HttpResponse.json | page.route(...) → route.fulfill(...) |
Key points:
- Vitest:
vi.setSystemTimeonly works aftervi.useFakeTimers(). Seed faker inbeforeEach. Set MSWonUnhandledRequest: 'error'so a missed stub fails loudly. - Playwright:
page.clock.install/setFixedTimemust run beforepage.goto.page.clockis the supported API — it exists; do not fall back to a hand-rolledDateoverride, and do not bump the timeout to "make it pass." - Pin locale/timezone/currency (
TZ=UTC,LANG) when the bug is locale-sensitive.
Avoid: jest.useFakeTimers('legacy') (and timers: 'legacy') — legacy fake timers are
deprecated and don't mock Date/Date.now, so the clock stays live and your "frozen"
repro still drifts. Modern timers are the default since Jest 27 — just call
jest.useFakeTimers() + jest.setSystemTime(), or vi.useFakeTimers() in Vitest. (Jest 30,
2025)
See references/determinism.md for the full Vitest (vi.useFakeTimers + vi.setSystemTime
faker.seed+ MSWsetupServer) and Playwright (page.clock+page.route) recipes, plus a 10-run determinism check.
Step 5: Write the failing regression test (red) first
You have a clean deterministic repro and the dev hasn't fixed it yet. Write the regression test now, before the fix, as TDD-for-bugs:
- Encode the minimal repro as a test that asserts the real expected value —
expect(total).toBe(2754). A tautological assertion that always passes proves nothing. - Run it and confirm it fails before the fix — it must be red first. A test that doesn't go red isn't exercising the bug.
- Commit the test (or stage it on the fix branch) so the test guards the fix in CI.
- After the dev's fix lands, re-run: it should flip to green. Same test, no edits.
Expected state: red before the fix, green after the fix.
Wrong moves that defeat the point — a test that genuinely fails until this bug is fixed: writing the test only after the fix already landed; disabling the failing test (pending markers, an always-true tautology, or removing the assertion) just to keep CI green; or fixing first and bolting a test on afterward. Keep the assertion live and let it go red.
See references/determinism.md for the deterministic test bodies the assertion sits in.
Step 6: Verify the fix actually fixes it
The dev pushed a fix and the regression test now passes. Green is necessary but not sufficient — a test can pass for an unrelated reason. Do not close on green alone or trust the dev's word. Run the validity check:
- Revert the fix temporarily (stash it, or
git stash/comment the fix line) and re-run the test. Confirm it still fails without the fix. That proves the test actually exercises this bug (the test catches the bug) — that it passes because of the fix, and is not passing for another reason unrelated to the defect. - Restore the fix and confirm green returns.
- Re-run deterministically several times (
--repeat-each/ a loop) to confirm the green is stable, not a lucky pass.
Only when the test is red-without-fix and green-with-fix, repeatably, is the fix verified. This revert-to-verify step is the whole point of the regression test and is the one most often left out.
Step 7: Flaky vs environment vs not-reproducible
You spent two hours and it won't reproduce for you, but it clearly happens for the user. Do not close as "cannot reproduce" immediately, do not assume flaky and quarantine, and do not conclude "doesn't repro on my machine so it isn't real." These are three distinct diagnoses with distinct evidence:
| Diagnosis | Discriminating evidence | What you do |
| Flaky | Same code, same env, passes and fails on the same commit — run --repeat-each 50 (or rerun the same command many times) in one environment and watch it flip | Find the non-determinism (time/RNG/network/race), make it deterministic (Step 4) |
| Environment-specific | Reproduces only under a different config — timezone, locale, viewport, OS, browser version, CI vs local — and is stable within that config | Match the user's environment: reproduce their timezone/locale/OS/browser, then minimize |
| Data-dependent | Reproduces only with the user's specific account/input | Get and replicate their data/fixture; the bug rides on the input, not the platform |
| Genuinely not reproducible | None of the above reproduces after matching env + data + repeat runs | Document what you tried (envs, run counts, data) and the negative result — don't silently close |
The step bare attempts miss: match the reported user environment and data before
judging — replicate their config (timezone, locale, OS, browser version) and reproduce
their env, then re-run. --repeat-each in the same env isolates true flakiness; cross-env
divergence points to environment-specific; input-dependence points to data-specific.
Step 8: Write the evidence back into the ticket
Reproduction and the committed regression test are done. Replace the vague original report with a structured block — do not paste the raw 14-step UI walkthrough, and do not just write "reproduced, closing." Include all seven elements:
- Minimal steps / repro command — the smallest path, not the walkthrough.
- Environment + build/commit — exact SHA and platform.
- Expected vs actual — the concrete numbers.
- Introducing commit — the offending commit from
git bisect. - Regression test — link to the committed test file and path.
- Evidence — logs, screenshot, trace, or artifact.
- Determinism notes — seed, frozen time, and stubs so anyone can re-run identically.
See references/ticket-writeback.md for the copy-paste Markdown structure.
Anti-Patterns
Avoid: fixing before reproducing, minimizing before confirming, removing multiple variables at once, manual bisect, exit 1 on build failure during bisect, live time/RNG/network in repro, writing test after fix, closing on green without revert-to-verify, closing "cannot reproduce" on first failure.
Full descriptions and corrections:
references/bisect-and-anti-patterns.md → "Anti-Patterns".
Failure Modes
| Symptom | Likely cause | Fix / check |
| Bisect lands on an obviously unrelated commit | Full suite or flaky failures marking commits bad | Switch to one targeted test; wrap with exit-125 skip + network stub |
| Repro passes locally, fails in CI (or vice versa) | Environment-specific (TZ, locale, OS, browser) | Pin TZ/LANG; match the failing environment (Step 7) |
| Test is green but you're not sure it catches the bug | Never ran it red | Revert the fix and confirm it fails (Step 6) |
vi.setSystemTime has no effect | Called before vi.useFakeTimers() | Call useFakeTimers() first |
page.clock time not applied to app startup | install/setFixedTime ran after page.goto | Move clock setup before navigation |
| Repro flips pass/fail run to run | Live time/RNG/network not pinned | Apply Step 4; confirm with --repeat-each 10 |
Verification
- The repro command/test fails on demand: run it 10× (
--repeat-each 10or a loop) and confirm it fails every time before the fix. git bisect run …terminates with "<sha>is the first bad commit" andgit bisect resetleaves you on the originalHEAD.- With the fix reverted the regression test exits non-zero; with the fix applied it exits 0.
- The ticket block contains all seven write-back elements (Step 8) — grep it for the commit SHA, the test path, and the determinism notes.
Done When
- A documented minimal reproduction exists — smallest steps or a single command — that fails on demand, verified failing across repeated runs.
- If the bug is a regression,
git bisecthas named the introducing commit SHA and it is recorded in the ticket. - The repro is deterministic: time frozen, RNG seeded, network stubbed — proven by 10 identical consecutive runs.
- A regression test is committed that was red before the fix and green after, and was confirmed to fail when the fix is reverted.
- The ticket carries the structured evidence block with all seven elements (minimal steps, environment/build, expected vs actual, introducing commit, regression-test link, evidence artifact, determinism notes); the original vague report is replaced, not left.
- If it did not reproduce, it is classified (flaky / environment-specific / data-dependent / not-reproducible) with the evidence that led there — never silently closed.
Related Skills
failure-analysis— Classify a failing test (product defect vs. outdated test). Classification decides what a failure is; come here to actually reproduce a defect and write the failing test. Classification hands off; bug-reproduction picks up.test-automation— Generate deterministic specs from understood flows. Use it when the source is a requirement; use this skill when the source is a defect and the test must first go red against the bug.test-reliability— Runtime self-healing and quarantine for a flaky test. When Step 7 diagnoses true flakiness, go there to stabilize or quarantine; here you only diagnose.- The bridge file
.agents/qa-project-context/<KEY>.md— stack, test runner, environment matrix, and known-flaky areas that shape every step above. Check it first. - systematic-debugging (external optional:
superpowers:systematic-debugging) — The general root-cause debugging loop once you have a deterministic repro; this skill produces that repro and the failing test that guards the eventual fix.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

