QA — Test → Fix → Verify

Wear two hats: QA engineer and bug-fix engineer. Drive the app the way a real user would — click everything, fill every form, push every state. When something breaks, fix it in source as a small atomic commit, then re-verify. Hand back a structured report with before/after evidence.

Gather context first

This skill ships standalone, so it can't assume your project's conventions, house style, or preferences the way it could for its author. Before doing the main work:

1. Auto-detect what you safely can from the repo — language/stack, base branch, build/test commands, existing config and docs. Never ask for something you can read for yourself.
2. Ask, don't assume, for the rest. Where an input, convention, or preference would change the result and you can't reliably detect it, ask ONE concise AskUserQuestion (put a sensible default first, labelled Recommended) instead of guessing. The user has less context than this skill's author assumed — a wrong silent default is worse than a quick question. Don't ask about things you can detect, and don't ask more than you need.

For this skill, confirm up front (only the items you can't already detect):

• The app under test — the URL/route to drive. Auto-detect from common local dev ports and any staging URL in the PR description or environment first; ask only if nothing responds or several candidate apps are up.
• Auth — whether the affected pages sit behind a login, and how to get in (credentials, a cookie file, or a live CDP browser session), plus whether 2FA/CAPTCHA will interrupt automation. Skip this question entirely when a real-browser CDP session is already attached.
• Test tier — Quick / Standard / Exhaustive (default Standard, Recommended), since the tier decides which severities get fixed versus deferred.
• Which flows matter most — when the diff is broad, the core surfaces to prioritise so depth lands where it counts.
• Where QA artifacts go — the .qa-reports/ directory and report filename, if this project keeps QA output somewhere else.

How this runs

Everything here runs inline in the current conversation by default. That keeps the interactive spine working — the clean-tree gate, the test-framework choice, the auth/2FA/CAPTCHA pauses, and the WTF-likelihood "stop and ask" checkpoint all need a direct answer from the user. The browser exploration (Phases 3-5) and the fix loop (Phases 7-9) are stateful too: one $B browser session carries cookies, viewport, and history, and each fix mutates the working tree with its own commit — so they proceed in order in this context.

Two parts are pure, non-interactive compute bursts and are the only candidates for parallelism:

Only fan either of these out after the user opts in. Before spawning any subagent or Workflow, stop and ask the user, for example:

This diff is large ([N changed files / big diff]). I can map it and QA it inline here (cheaper, slower) or fan out [M] parallel subagents (faster, more tokens). Which do you want?

Spawn subagents/workflows only after an explicit yes. If the user declines or doesn't answer, do the whole job inline. When you do fan out, remember the constraints: Workflow agent() calls run in the background, so they cannot call AskUserQuestion, talk to the user, or write files. Every gate (clean tree, framework bootstrap, auth, CAPTCHA, WTF-likelihood stop), every browser action, every fix commit, and all file writes — the QA report, baseline.json, screenshots, regression tests, TODOS.md updates — stay in the main context. A workflow only does headless analysis (map a diff, trace source, produce test cases) and returns structured results; you act on them and write the files. Set model: 'opus' on every agent() call, and pass data in via args (workflow scripts have no Date.now/Math.random/filesystem access — timestamps, the report date, and every write happen in the main context).

Arguments

Check $ARGUMENTS:

• A target URL → run full mode against it (e.g. http://localhost:3000, https://staging.example.com).
• A tier hint — --quick / --exhaustive → set the fix tier.
• A mode hint — --regression <baseline.json> → regression mode against that baseline.
• A scope hint — "Focus on the billing page" → diff-scope or page-scope the run.
• A PR number / commit ref / feature name → use it to scope the diff and label the report.
• Empty + on a feature branch → diff-aware mode (the common case; the user just shipped code and wants to confirm it works).

Setup

Parse the request for these parameters:

Tiers decide which issues get fixed:

• Quick: critical + high severity only
• Standard: + medium severity (default)
• Exhaustive: + low/cosmetic severity

If no tier hint arrived in $ARGUMENTS, default to Standard, but confirm the depth with the user when the run looks large enough that the tier would meaningfully change how much gets fixed.

No URL given and on a feature branch: enter diff-aware mode automatically. This is the most common entry point — someone just pushed work to a branch and wants it verified.

Require a clean working tree:

git status --porcelain

If the output is non-empty, STOP and use AskUserQuestion:

"Your working tree has uncommitted changes. QA needs a clean tree so each bug fix lands as its own atomic commit."

A) Commit my changes — commit everything now with a descriptive message, then start QA B) Stash my changes — stash, run QA, pop the stash after C) Abort — I'll clean up manually

RECOMMENDATION: Choose A — uncommitted work should be captured as a commit before QA adds its own fix commits.

Browser automation tool ($B):

$B is whatever browser-driving CLI the environment exposes — it offers commands like goto, snapshot, console, click, and fill. Detect one:

command -v browse >/dev/null 2>&1 && B=browse
[ -z "$B" ] && [ -x "./scripts/browse" ] && B="./scripts/browse"

You can also wire $B to a Playwright/Puppeteer MCP server or a project-specific browser script. If nothing is available, fall back to source-code QA: read component source, trace data flow, predict behavior, and document manual test cases the user can run themselves.

Detect CDP mode (when $B is attached to a real browser):

$B status 2>/dev/null | grep -q "Mode: cdp" && echo "CDP_MODE=true" || echo "CDP_MODE=false"

If CDP_MODE=true: skip cookie-import prompts (the real browser already has cookies), skip user-agent overrides, skip headless workarounds. Real auth sessions are already in place.

Create output directories:

.qa-reports/ is the default location for reports and screenshots. If the project already keeps QA artifacts somewhere else, or the user prefers another path, use that instead.

mkdir -p .qa-reports/screenshots

Step 0: Detect base branch

1. gh pr view --json baseRefName -q .baseRefName — use it if it succeeds
2. gh repo view --json defaultBranchRef -q .defaultBranchRef.name — use it if it succeeds
3. Fallback: git symbolic-ref refs/remotes/origin/HEAD 2>/dev/null | sed 's|refs/remotes/origin/||'
4. Final fallback: main — if it falls through to this fallback the base is ambiguous, so confirm with the user rather than silently assuming main

Test Framework Bootstrap

Detect the project's test framework before running QA. If none exists and the project is non-trivial, offer to bootstrap one so regression tests can be generated in Phase 8e.5.

# Detect project runtime
[ -f Gemfile ] && echo "RUNTIME:ruby"
[ -f package.json ] && echo "RUNTIME:node"
[ -f requirements.txt ] || [ -f pyproject.toml ] && echo "RUNTIME:python"
[ -f go.mod ] && echo "RUNTIME:go"
[ -f Cargo.toml ] && echo "RUNTIME:rust"
[ -f composer.json ] && echo "RUNTIME:php"
[ -f mix.exs ] && echo "RUNTIME:elixir"
# Detect sub-frameworks
[ -f Gemfile ] && grep -q "rails" Gemfile 2>/dev/null && echo "FRAMEWORK:rails"
[ -f package.json ] && grep -q '"next"' package.json 2>/dev/null && echo "FRAMEWORK:nextjs"
# Check existing test infrastructure
ls jest.config.* vitest.config.* playwright.config.* .rspec pytest.ini pyproject.toml phpunit.xml 2>/dev/null
ls -d test/ tests/ spec/ __tests__/ cypress/ e2e/ 2>/dev/null
# Opt-out marker
[ -f .qa/no-test-bootstrap ] && echo "BOOTSTRAP_DECLINED"

Test framework detected: read 2-3 existing test files to learn the conventions (naming, imports, assertion style, setup patterns). Hold that as prose context for Phase 8e.5. Skip the rest of bootstrap.

BOOTSTRAP_DECLINED present: skip the rest of bootstrap.

No runtime detected: AskUserQuestion — "What runtime are you using?" Options: Node.js/TypeScript, Ruby/Rails, Python, Go, Rust, PHP, Elixir, This project doesn't need tests. If the user picks "doesn't need tests" → write .qa/no-test-bootstrap and continue.

Runtime detected but no test framework — bootstrap:

Research best practices

WebSearch for current guidance:

• "[runtime] best test framework {current year}"
• "[framework A] vs [framework B] comparison"

If WebSearch is unavailable, use this table:

Framework selection

AskUserQuestion offering the recommended primary + alternative + a skip option. If the user skips, write .qa/no-test-bootstrap and continue without tests.

Install, configure, verify

1. Install the chosen packages
2. Create a minimal config file
3. Create the directory structure (test/, spec/, etc.)
4. Generate 3-5 real tests for existing code, prioritized by risk (error handlers > business logic with conditionals > API endpoints > pure functions). Never expect(x).toBeDefined() — test what the code actually DOES.
5. Run each test. Passes → keep. Fails → fix once. Still fails → delete silently.
6. Run the full suite to confirm green.
7. If CI is detected (.github/), create .github/workflows/test.yml with the verified test command.
8. Write TESTING.md with the framework name, run command, and conventions.
9. Append a ## Testing section to CLAUDE.md if one isn't already there.
10. Commit: git commit -m "chore: bootstrap test framework ({name})"

If installation fails → debug once. Still failing → revert with git checkout -- package.json package-lock.json (or equivalent), warn the user, and continue without tests.

Never import secrets, API keys, or credentials into test files.

Test Plan Context

Before falling back to raw git-diff heuristics, look for a richer test plan:

1. Conversation context: if a prior /plan-review or /scope-review produced test-plan output in this conversation, use it directly.
2. Project-saved test plans: if the project keeps test-plan notes for the current branch (e.g. a doc left by a prior /plan-review run), read the most recent one.
3. Fall back to git-diff analysis only when neither exists.

Modes

Diff-aware (automatic on a feature branch with no URL)

Primary mode for developers verifying their own work:

1. Analyze the branch diff:

   git diff <base>...HEAD --name-only
   git log <base>..HEAD --oneline

2. Map changed files to affected pages/routes:

   • Controller/route files → the URL paths they serve
   • View/template/component files → the pages that render them
   • Model/service files → the pages that use them (check referencing controllers)
   • CSS/style files → the pages that include them
   • API endpoints → test directly with $B js "await fetch('/api/...')"
   • Static pages (markdown, HTML) → navigate directly

   This mapping is independent per file. By default, work through the changed files one at a time in this context and union the affected targets. If the diff is large and the user opted into subagents, fan it out as the diff-mapping workflow below, then assemble the affected-pages set here.

   If no obvious pages/routes surface: do not skip browser testing. Drop to Quick mode — open the homepage, follow the top 5 nav targets, and check the console. Backend/config/infra changes still change app behavior, so always verify something.

3. Detect the running app — probe common local dev ports:

   $B goto http://localhost:3000 2>/dev/null && echo "Found app on :3000" || \
   $B goto http://localhost:4000 2>/dev/null && echo "Found app on :4000" || \
   $B goto http://localhost:8080 2>/dev/null && echo "Found app on :8080"

   If nothing responds, check the PR description or environment for a staging URL. Otherwise ask the user.
4. Test each affected page/route:
   • Navigate to the page
   • Take a screenshot
   • Check the console for errors
   • If the change is interactive (forms, buttons, flows), test it end-to-end
   • Use snapshot -D before and after actions to confirm the effect
5. Cross-reference commit messages and the PR description to understand intent.
6. Check TODOS.md for known bugs touching the changed files, and add any matches to the test plan.
7. Report findings scoped to the branch changes.

Diff → affected-pages mapping — optional parallel workflow

Default: map the files inline, one after another. Only if the user opted into subagents and the diff touches many files, fan out the mapping so each changed file is analyzed by its own agent in parallel, then merge here. Read-only analysis; no shared state.

export const meta = {
  name: 'qa-diff-map',
  description: 'Parallel per-file analysis mapping each changed file to affected pages/routes',
  phases: [{ title: 'Map' }],
}
const FILE_SCHEMA = {
  type: 'object',
  properties: {
    file: { type: 'string' },
    kind: { type: 'string' },                                        // controller | view | model | style | api | static | other
    affected: {
      type: 'array',
      items: {
        type: 'object',
        properties: {
          target: { type: 'string' },                               // URL path or route
          how: { type: 'string' },                                  // navigate | fetch | renders-component | includes-style
          interactive: { type: 'boolean' },                         // forms/buttons/flows present
        },
        required: ['target', 'how'],
      },
    },
    notes: { type: 'string' },
  },
  required: ['file', 'kind', 'affected'],
}
const maps = await parallel(args.files.map(f => () =>
  agent(
    `You are mapping ONE changed file to the pages/routes it affects. Do NOT ask anything — ` +
    `return findings only.\n\n` +
    `CHANGED FILE: ${f.path}\nDIFF:\n${f.diff}\n\n` +
    `Classify the file (controller/route, view/template/component, model/service, CSS/style, ` +
    `API endpoint, static page, or other). Then determine every URL path or route it affects and ` +
    `how to reach it: controllers/routes → the URL paths they serve; views/components → the pages ` +
    `that render them; models/services → the pages that use them (trace referencing controllers); ` +
    `CSS → the pages that include it; API endpoints → the fetch URL. Mark targets that involve ` +
    `forms/buttons/flows as interactive.`,
    { label: `map:${f.key}`, phase: 'Map', model: 'opus', schema: FILE_SCHEMA }
  )
))
return maps.filter(Boolean)

Pass args: { files: [{ key, path, diff }, ...] }. When the workflow returns, union the affected targets here (dedupe by URL); if the union is empty, fall back to Quick mode as above. If the workflow fails or times out, map the files inline.

Full (default when a URL is provided)

Systematic exploration. Visit every reachable page. Document 5-10 well-evidenced issues. Produce a health score. 5-15 minutes depending on app size.

Quick (--quick)

30-second smoke test. Visit the homepage + top 5 nav targets. Check: does the page load? Console errors? Broken links? Produce a health score. No detailed issue write-ups.

Regression (--regression <baseline>)

Run full mode, then load baseline.json from a previous run. Diff them: which issues are fixed? Which are new? What's the score delta? Append a regression section to the report.

Workflow

Phase 1: Initialize

1. Find the browser tool $B (or fall back to source-code QA)
2. Create output directories
3. Start a timer for duration tracking

Phase 2: Authenticate (if needed)

If the user gave auth credentials:

$B goto <login-url>
$B snapshot -i                    # find the login form
$B fill @e3 "user@example.com"
$B fill @e4 "[REDACTED]"          # NEVER put a real password in the report
$B click @e5                       # submit
$B snapshot -D                     # confirm login succeeded

If the user gave a cookie file:

$B cookie-import cookies.json
$B goto <target-url>

If 2FA/OTP is required: ask the user for the code and wait.

If a CAPTCHA blocks you: tell the user — "Please complete the CAPTCHA in the browser, then tell me to continue."

Phase 3: Orient

Get a map of the application:

$B goto <target-url>
$B snapshot -i -a -o ".qa-reports/screenshots/initial.png"
$B links                          # map the navigation structure
$B console --errors               # any errors on landing?

Detect the framework (note it in report metadata):

• __next in HTML or _next/data requests → Next.js
• csrf-token meta tag → Rails
• wp-content in URLs → WordPress
• Client-side routing with no page reloads → SPA

For SPAs: links may return little because navigation is client-side. Use snapshot -i to find nav elements instead.

Phase 4: Explore

Walk the pages systematically. At each page:

$B goto <page-url>
$B snapshot -i -a -o ".qa-reports/screenshots/page-name.png"
$B console --errors

Then run the per-page checklist:

1. Visual scan — read the annotated screenshot for layout problems
2. Interactive elements — click buttons, links, controls. Do they work?
3. Forms — fill and submit. Test empty, invalid, and edge-case input
4. Navigation — check every path in and out
5. States — empty, loading, error, overflow
6. Console — any new JS errors after interacting?
7. Responsiveness — check the mobile viewport when relevant:

   $B viewport 375x812
   $B screenshot ".qa-reports/screenshots/page-mobile.png"
   $B viewport 1280x720

Depth judgment: spend more time on core surfaces (homepage, dashboard, checkout, search) and less on secondary ones (about, terms, privacy).

Quick mode: only the homepage + top 5 nav targets. Skip the checklist — just check: loads? Console errors? Visibly broken links?

If $B is unavailable (source-code fallback): for each affected page, read the component source, trace its data flow (entry → validation → state → API → render), and document manual test cases using the Phase 5 template. Also produce the State Coverage Check below. By default, work through the pages one at a time in this context. If there are several pages and the user opted into subagents, fan this out as the source-code-QA workflow below, then write up the returned test cases and State Coverage tables here.

State Coverage Check (source-code mode)

For each UI feature on the affected page, confirm the code handles every state:

FEATURE              | LOADING | EMPTY | ERROR | SUCCESS | PARTIAL
---------------------|---------|-------|-------|---------|--------
[feature name]       | ?       | ?     | ?     | ?       | ?

Mark each "?" as:

• HANDLED: the code explicitly handles this state with appropriate UI
• MISSING: the code does not handle this state (the user sees blank/broken UI)
• PARTIAL: handled but with issues (e.g., a loading spinner shows but never clears)

Source-code-mode QA — optional parallel workflow

Default: analyze the affected pages one at a time in this context. Only if $B is unavailable, several pages are affected, and the user opted into subagents, run the per-page source analysis in parallel — each agent reads one page's source, traces its data flow, and returns manual test cases plus the State Coverage table. Read-only; no shared state. Then write the report here.

export const meta = {
  name: 'qa-source-qa',
  description: 'Parallel per-page source-code QA — trace data flow, produce test cases + state coverage',
  phases: [{ title: 'Trace' }],
}
const PAGE_SCHEMA = {
  type: 'object',
  properties: {
    page: { type: 'string' },
    sourceFiles: { type: 'array', items: { type: 'string' } },
    dataFlow: { type: 'string' },                                    // entry → validation → state → API → render
    testCases: {
      type: 'array',
      items: {
        type: 'object',
        properties: {
          id: { type: 'string' },                                   // TC-NNN
          precondition: { type: 'string' },
          steps: { type: 'array', items: { type: 'string' } },
          expected: { type: 'string' },
          verify: { type: 'string' },
          risk: { type: 'string' },
        },
        required: ['id', 'steps', 'expected'],
      },
    },
    stateCoverage: {
      type: 'array',
      items: {
        type: 'object',
        properties: {
          feature: { type: 'string' },
          loading: { type: 'string' },                              // HANDLED | MISSING | PARTIAL
          empty: { type: 'string' },
          error: { type: 'string' },
          success: { type: 'string' },
          partial: { type: 'string' },
        },
        required: ['feature', 'loading', 'empty', 'error', 'success', 'partial'],
      },
    },
    suspectedBugs: {
      type: 'array',
      items: {
        type: 'object',
        properties: {
          title: { type: 'string' },
          severity: { type: 'string' },                             // Critical | High | Medium | Low
          fileLine: { type: 'string' },                             // path/to/file.ts:LINE
          impact: { type: 'string' },
          suggestedFix: { type: 'string' },
        },
        required: ['title', 'severity', 'fileLine', 'impact'],
      },
    },
  },
  required: ['page', 'sourceFiles', 'dataFlow', 'testCases', 'stateCoverage'],
}
const pages = await parallel(args.pages.map(p => () =>
  agent(
    `No browser is available, so QA this page from source only. Do NOT ask anything — ` +
    `return findings only.\n\n` +
    `PAGE: ${p.target}\nKNOWN SOURCE FILES: ${p.files}\n\n` +
    `Read the component source, trace the data flow (entry → validation → state → API → render), ` +
    `and produce: (1) manual test cases (id TC-NNN, precondition, steps, expected, verify, risk); ` +
    `(2) a State Coverage table marking LOADING/EMPTY/ERROR/SUCCESS/PARTIAL as HANDLED, MISSING, or ` +
    `PARTIAL for each UI feature; (3) any suspected bugs you can see in source with file:line, ` +
    `severity, impact, and a suggested fix.`,
    { label: `trace:${p.key}`, phase: 'Trace', model: 'opus', schema: PAGE_SCHEMA }
  )
))
return pages.filter(Boolean)

Pass args: { pages: [{ key, target, files }, ...] }. When the workflow returns, write each test case and State Coverage table into the report here (Phase 5 / Phase 10 formats), and feed suspectedBugs into the issue list. If the workflow fails or times out, analyze the pages one at a time in this context.

Phase 5: Document

Document each issue the moment you find it — don't batch.

Two evidence tiers:

Interactive bugs (broken flows, dead buttons, form failures):

1. Screenshot before the action
2. Perform the action
3. Screenshot the result
4. Use snapshot -D to show what changed
5. Write repro steps that reference the screenshots

$B screenshot ".qa-reports/screenshots/issue-001-step-1.png"
$B click @e5
$B screenshot ".qa-reports/screenshots/issue-001-result.png"
$B snapshot -D

Static bugs (typos, layout issues, missing images):

1. One annotated screenshot showing the problem
2. Describe what's wrong

Source-code-mode bugs: file path + line number + the code snippet showing the bug + an impact description.

Write each issue to the report immediately. Use this format:

### QA-NNN: [Title]

**Severity:** Critical / High / Medium / Low
**Category:** Functional / Visual / UX / Performance / Data Safety / A11y
**Page:** [URL path]
**File:** `path/to/file.ts:LINE` (if code-level)

**Description:** What is wrong
**Impact:** What the user experiences
**Repro steps:**
  1. Navigate to [page]
  2. Do [action]
  3. Observe [problem]

**Expected:** [correct behavior]
**Actual:** [broken behavior]

**Evidence:** [screenshot paths or code snippet]
**Suggested fix:** [recommended code change]

Manual test cases (for issues that need human verification):

MANUAL TEST CASE: TC-001
========================
Page: [URL]
Precondition: [state required]
Steps:
  1. [step]
  2. [step]
Expected: [behavior]
Verify: [observation]
Risk: [what could go wrong]

Phase 6: Wrap Up (compute baseline health score)

1. Compute the health score with the rubric below
2. Write "Top 3 Things to Fix" — the 3 highest-severity issues
3. Console health summary — aggregate every console error seen across pages
4. Update the severity counts in the summary table
5. Fill in report metadata — date, duration, pages visited, screenshot count, framework
6. Save the baseline — write .qa-reports/baseline.json:

   {
     "date": "YYYY-MM-DD",
     "url": "<target>",
     "healthScore": N,
     "issues": [{ "id": "QA-001", "title": "...", "severity": "...", "category": "..." }],
     "categoryScores": { "console": N, "links": N, ... }
   }

Regression mode: after writing the baseline, load the previous baseline file and compare:

• Health-score delta
• Issues fixed (in the old baseline but not current)
• New issues (in current but not the old baseline)
• Append a regression section to the report

Record the baseline health score at the end of Phase 6.

Health Score Rubric

Compute each category score (0-100), then take the weighted average.

Console (weight: 15%)

• 0 errors → 100
• 1-3 errors → 70
• 4-10 errors → 40
• 10+ errors → 10

Links (weight: 10%)

• 0 broken → 100
• Each broken link → -15 (minimum 0)

Per-Category Scoring (Visual, Functional, UX, Content, Performance, Accessibility, Data Safety)

Each category starts at 100. Deduct per finding:

• Critical issue → -25
• High issue → -15
• Medium issue → -8
• Low issue → -3

Minimum 0 per category.

Weights

Final Score

score = Σ (category_score × weight)

Phase 7: Triage

Sort every discovered issue by severity. Decide what to fix based on tier:

• Quick: fix critical + high. Mark medium/low as "deferred."
• Standard: fix critical + high + medium. Mark low as "deferred."
• Exhaustive: fix everything, including cosmetic.

Mark anything that can't be fixed from source (third-party widget bugs, infrastructure issues) as "deferred" regardless of tier.

Phase 8: Fix Loop

The fix loop is stateful and runs in this context — each fix mutates the working tree and gets its own atomic commit, so it is sequential by construction. For each fixable issue, in severity order:

8a. Locate source

# Grep for error messages, component names, route definitions
# Glob for file patterns matching the affected page

• Find the source file(s) responsible for the bug
• ONLY touch files directly related to the issue

8b. Fix

• Read the source, understand the context
• Make the minimal fix — the smallest change that resolves the issue
• Do NOT refactor surrounding code, add features, or "improve" unrelated things

For complex fixes (multi-file, logic changes): hand off to /qa-fix with the specific issue details and let it trace, fix, and verify.

8c. Commit

git add <only-changed-files>
git commit -m "fix(qa): QA-NNN — short description"

• One commit per fix. Never bundle.
• Message format: fix(qa): QA-NNN — short description

8d. Re-test

• Navigate back to the affected page
• Take a before/after screenshot pair
• Check the console for errors
• Use snapshot -D to confirm the effect

$B goto <affected-url>
$B screenshot ".qa-reports/screenshots/issue-NNN-after.png"
$B console --errors
$B snapshot -D

8e. Classify

• verified: the re-test confirms the fix works and introduces no new errors
• best-effort: fix applied but couldn't be fully verified (needs auth state, external service)
• reverted: a regression appeared → git revert HEAD → mark the issue "deferred"

8e.5. Regression Test

Skip if: the classification isn't "verified", OR the fix is purely visual/CSS with no JS behavior, OR no test framework was detected AND the user declined bootstrap.

1. Study the project's existing test patterns:

Read 2-3 test files closest to the fix. Match exactly: file naming, imports, assertion style, describe/it nesting, setup/teardown. The regression test should read as if the same developer wrote it.

2. Trace the bug's codepath, then write a regression test:

Before writing, trace:

• What input/state triggered the bug? (exact precondition)
• What codepath did it follow? (branches, function calls)
• Where did it break? (exact line/condition)
• What other inputs hit the same codepath? (edge cases)

The test MUST:

• Set up the precondition that triggered the bug
• Perform the action that exposed it
• Assert the correct behavior (NOT "it renders" or "it doesn't throw")
• Cover adjacent edge cases if found (null input, empty array, boundary values)
• Carry an attribution header:

  // Regression: QA-NNN — {what broke}
  // Found by qa on {YYYY-MM-DD}
  // Report: .qa-reports/qa-report-{domain}-{date}.md

Test type decision:

• Console error / JS exception / logic bug → unit or integration test
• Broken form / API failure / data flow bug → integration test with request/response
• Visual bug with JS behavior (broken dropdown, animation) → component test
• Pure CSS → skip

Generate unit tests. Mock every external dependency (DB, API, Redis, file system).

Use auto-incrementing names to avoid collisions: check existing {name}.regression-*.test.{ext} files and take max number + 1.

3. Run only the new test file:

{detected test command} {new-test-file}

4. Evaluate:

• Passes → commit: git commit -m "test(qa): regression test for QA-NNN — {desc}"
• Fails → fix the test once. Still failing → delete it and defer.
• Taking >2 min of exploration → skip and defer.

5. WTF-likelihood exclusion: test commits don't count toward the heuristic.

8f. Self-Regulation (STOP AND EVALUATE)

Every 5 fixes (or after any revert), compute WTF-likelihood:

WTF-LIKELIHOOD:
  Start at 0%
  Each revert:                +15%
  Each fix touching >3 files: +5%
  After fix 15:               +1% per additional fix
  All remaining Low severity: +10%
  Touching unrelated files:   +20%

If WTF > 20%: STOP immediately. Show the user what you've done so far. Ask whether to continue (AskUserQuestion).

Hard cap: 50 fixes. After 50 fixes, stop regardless of what remains.

Phase 9: Final QA

After every fix is applied:

1. Re-run QA on all affected pages
2. Compute the final health score
3. If the final score is WORSE than the baseline: WARN prominently — something regressed

Phase 10: Report

Write to .qa-reports/qa-report-{domain}-{YYYY-MM-DD}.md:

# QA Report — [branch name]

**Date:** YYYY-MM-DD
**Branch:** [branch]
**Base:** [base branch]
**Tier:** Quick / Standard / Exhaustive
**Mode:** Diff-aware / Full / Quick / Regression
**Duration:** Nm
**Framework:** [detected framework]

---

## Summary

| Metric | Value |
|--------|-------|
| Pages/routes tested | N |
| Issues found | N |
| Issues fixed | N (verified: X, best-effort: Y, reverted: Z) |
| Issues deferred | N |
| Health score (before) | X/100 |
| Health score (after) | Y/100 |

## Top 3 Things to Fix

1. **[QA-NNN]** [title] — [one-line impact]
2. **[QA-NNN]** [title] — [one-line impact]
3. **[QA-NNN]** [title] — [one-line impact]

## Console Health

[Aggregated console errors across all pages]

## Manual Test Cases

[List of test cases for manual verification]

## All Issues

[Full issue details, per-issue additions beyond the base template:]
- **Fix Status:** verified / best-effort / reverted / deferred
- **Commit SHA:** [if fixed]
- **Files Changed:** [if fixed]
- **Before/After screenshots:** [if fixed]

## Ship Readiness

| Check | Status |
|-------|--------|
| Critical issues remaining | N |
| High issues remaining | N |
| Health score | X/100 |
| **Verdict** | SHIP / SHIP WITH CAUTION / DO NOT SHIP |

**Verdict rules:**
- 0 critical + 0 high → SHIP
- 0 critical + some high → SHIP WITH CAUTION
- Any critical remaining → DO NOT SHIP

## PR Summary

> "QA found N issues, fixed M, health score X → Y."

If a .qa-reports/qa-report-*.md already exists for today, append the new report below the existing content.

Phase 11: TODOS.md Update

If the repo has a TODOS.md:

1. New deferred bugs → add as TODOs with severity, category, and repro steps
2. Fixed bugs that were in TODOS.md → annotate with "Fixed by qa on {branch}, {date}"

Framework-Specific Checks

Next.js

• Watch the console for hydration errors (Hydration failed, Text content did not match)
• Monitor _next/data requests; 404s mean broken data fetching
• Test client-side navigation (click links, don't just goto) to catch routing issues
• Check for CLS (Cumulative Layout Shift) on pages with dynamic content
• useRouter null safety (query params undefined on first render)
• Missing key props in dynamic lists

Rails

• N+1 query warnings in the console (development mode)
• CSRF token presence in forms
• Turbo/Stimulus integration — do page transitions feel smooth?
• Flash messages appearing and dismissing correctly

WordPress

• Plugin conflicts (JS errors from different plugins)
• Admin bar visibility for logged-in users
• REST API endpoints (/wp-json/)
• Mixed-content warnings

General SPA (React, Vue, Angular)

• Use snapshot -i for navigation — links misses client-side routes
• Stale state (navigate away and back — does the data refresh?)
• Browser back/forward — does the app handle history correctly?
• Memory leaks (watch the console after extended use)

React + Redux

• Selectors handle the initial state (before data loads)
• Dispatch calls use the correct action types
• Async mutations cover every path (loading, success, error)
• useEffect dependency arrays are complete
• Optional-chaining gaps on Redux state

API layer (axios/fetch)

• Error responses checked (status codes, error_code fields)
• Loading states set/cleared on every path (including error)
• Race conditions (component unmounts during an API call)
• Missing try/catch around API calls

i18n

• Hardcoded strings that should use t() or a translation function
• Missing translation keys
• Locale fallback behavior

Important Rules

1. Repro is everything. Every issue needs at least one screenshot (or file:line in source-code mode). No exceptions.
2. Verify before documenting. Retry the issue once to confirm it's reproducible, not a fluke.
3. Never include credentials. Write [REDACTED] for passwords in repro steps.
4. Write incrementally. Append each issue to the report as you find it. Don't batch.
5. Check the console after every interaction. JS errors that don't surface visually are still bugs.
6. Test like a user. Use realistic data. Walk complete workflows end-to-end.
7. Depth over breadth. 5-10 well-documented issues with evidence beat 20 vague descriptions.
8. Never delete output files. Screenshots and reports accumulate on purpose.
9. Show screenshots to the user. After every $B screenshot, $B snapshot -a -o, or $B responsive command, Read the output file(s) so the user sees them inline. For responsive (3 files), Read all three. Critical — without it, screenshots are invisible to the user.
10. Never refuse to use the browser. Invoking qa is a request for browser-based testing. Don't suggest evals or unit tests as a substitute. Even when the diff looks UI-free, backend changes affect behavior — open the browser and test.
11. Clean working tree required. If dirty, use AskUserQuestion to offer commit/stash/abort before proceeding.
12. One commit per fix. Never bundle.
13. Minimal fixes only. Don't refactor.
14. Only modify tests when generating regression tests in Phase 8e.5. Never modify CI config. Never modify existing tests — only create new test files.
15. Revert on regression. If a fix makes things worse: git revert HEAD immediately.
16. Self-regulate. Follow the WTF-likelihood heuristic. When in doubt, stop and ask.
17. Always produce the report. Even when no issues are found.
18. Ship-readiness verdict is mandatory. Never end without one.
19. Stateful and interactive work stays in this context; only headless analysis may fan out (and only after the user opts in). Never put an AskUserQuestion gate, a browser action, a fix commit, or any file write inside a Workflow agent() — background agents can't talk to the user or touch the filesystem.

Output Structure

.qa-reports/
├── qa-report-{domain}-{YYYY-MM-DD}.md    # Structured report
├── screenshots/
│   ├── initial.png                        # Landing page annotated screenshot
│   ├── issue-001-step-1.png               # Per-issue evidence
│   ├── issue-001-result.png
│   ├── issue-001-before.png               # Before fix (if fixed)
│   ├── issue-001-after.png                # After fix (if fixed)
│   └── ...
└── baseline.json                          # For regression mode

Report filenames use the domain and date: qa-report-myapp-com-2026-03-12.md

Next Steps — Review Chaining

• /qa-fix — for source-level QA of complex multi-file fixes that need deeper tracing than this skill's fix loop applies.
• /qa-report — when the user wants a report-only assessment of someone else's branch without modifying it.
• /pre-merge-review — run as a separate pre-landing structural gate after fixes land (it's a sibling skill — invoke /pre-merge-review directly, not as a subagent spawn).
• /ship — to merge, test, push, and open the PR once the ship-readiness verdict is SHIP.