agentleFS
napnap11napnap11/claude-skills/qa-report/SKILL.md

qa-report

Skill1 starsChanged 5 months agoMITView on GitHub

Read-only QA pass for a web app. Drives the product like a real user, captures evidence, and assembles a structured report with a health score, repro steps, and a ship-readiness verdict. Never edits source, never commits, never proposes fixes — strictly observe and document. Diff-aware mode narrows testing to the files changed on a feature branch. For the full test-fix-verify loop, reach for /qa or /qa-fix instead. Use when the user says "qa report", "/qa-report", "report-only QA", "QA report only", "don't fix, just report", "assess before deciding to fix", "ship readiness report", or asks to verify a branch read-only without modifying it.

7 other projects use a version of this.

Adopted by

Most recently taken up by

The file

Show its rules and text

We split this skill into the rules it gives an agent

31 rules. 0 are common, 18 are distinctive, and 13 are very specific to this project.

  • 0

    Common

    Most of the 6 edited versions we read keep these.

  • 18

    Distinctive

    This author's own choices. Worth a look.

  • 13

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    You observe and document. You never fix: No source edits, no commits — the constraint is deliberate.

    line 21
  2. Very specific:

    Read $ARGUMENTS and route on what it contains:

    line 50
  3. Distinctive:

    If no browser CLI is available, fall back to source-code review mode: read component source, trace data flow, predict behavior, and …

    line 83
  4. Very specific:

    If CDP_MODE=true: skip cookie-import prompts (the real browser already carries cookies), skip user-agent overrides, skip headless-detection …

    line 89
  5. Distinctive:

    Before dropping to raw git-diff heuristics, look for a richer test plan to scope against:

    line 110
  6. Distinctive:

    Map changed files to affected pages/routes: Do this inline by default.

    line 129
  7. Very specific:

    No obvious pages/routes surface?: Don't skip browser testing.

    line 137
  8. Distinctive:

    If none respond, check the PR description or environment for a staging URL.

    line 145
  9. Distinctive:

    Check the console for errors

    line 150
  10. Distinctive:

    If the change is interactive (forms, buttons, flows), walk it end-to-end

    line 151
  11. Very specific:

    Use snapshot -D before and after actions to confirm the effect

    line 152
  12. Very specific:

    Run Full mode, then load baseline.json from a previous run.

    line 173
  13. Distinctive:

    Create the output directories

    line 314
  14. Distinctive:

    Start a timer for duration tracking

    line 315
  15. Distinctive:

    Read the project's CLAUDE.md and any project memory files for conventions

    line 316
  16. Distinctive:

    2FA/OTP required: ask the user for the code and wait.

    line 338
  17. Distinctive:

    Write each issue the moment you find it — don't batch.

    line 409
  18. Very specific:

    Use snapshot -D to show what changed

    line 417
  19. Distinctive:

    Write repro steps that reference the screenshots

    line 418
  20. Distinctive:

    Regression mode: after writing the baseline, load the previous baseline and compare:

    line 491
  21. Very specific:

    Write the final report to .qa-reports/qa-report-{domain}-{YYYY-MM-DD}.md (or the output location the user confirmed up front — see Gather …

    line 497
  22. Very specific:

    If .qa-reports/qa-report-*.md already exists for today, append the new report below the existing content.

    line 570
  23. Very specific:

    Test client-side navigation (click links, don't just goto) — this catches routing issues

    line 622
  24. Distinctive:

    Check for CLS (Cumulative Layout Shift) on pages with dynamic content

    line 623
  25. Very specific:

    Use snapshot -i for navigation — links misses client-side routes

    line 640
  26. Distinctive:

    Repro is everything: Every issue needs at least one screenshot (or file:line in source-code mode).

    line 670
  27. Very specific:

    Never include credentials: Write [REDACTED] for passwords in repro steps.

    line 672
  28. Distinctive:

    Test like a user: Use realistic data. Walk complete workflows end-to-end.

    line 675
  29. Very specific:

    Show screenshots to the user: After every $B screenshot, $B snapshot -a -o, or $B responsive command, Read the output file(s) so the user …

    line 678
  30. Distinctive:

    The ship-readiness verdict is mandatory: Never finish without one.

    line 681
  31. Distinctive:

    When the issues need fixing:

    line 706

“Common” is measured: kept by at least 6 in 10 of the 6 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

QA Report — Read-Only Testing

You are a QA engineer. Drive the app the way a real user would: open every page, click every control, submit every form, push every state. Capture evidence as you go and assemble it into a structured report.

You observe and document. You never fix. No source edits, no commits — the constraint is deliberate. When the bugs need fixing, hand off to /qa (the full test → fix → verify loop) or /qa-fix (targeted source-level fixes).

Gather context first

This skill ships standalone, so it can't assume your project's conventions, house style, or preferences the way it could for its author. Before doing the main work:

  1. Auto-detect what you safely can from the repo — language/stack, base branch, build/test commands, existing config and docs. Never ask for something you can read for yourself.
  2. Ask, don't assume, for the rest. Where an input, convention, or preference would change the result and you can't reliably detect it, ask ONE concise AskUserQuestion (put a sensible default first, labelled Recommended) instead of guessing. The user has less context than this skill's author assumed — a wrong silent default is worse than a quick question. Don't ask about things you can detect, and don't ask more than you need.

For this skill, confirm up front (only the items you can't already detect):

  • Where the app is running — the target URL or local dev port. Probe the usual ports first (:3000, :4000, :8080); only ask if none respond and no staging URL surfaces in the PR/env.
  • Auth requirements — does the testing need a signed-in session, and if so via credentials or a cookie import? (Skip when CDP mode is detected — the real browser already carries cookies.)
  • Scope & depth (mode/tier) — which pages/flows matter most, and whether to run diff-aware (default on a feature branch), Full, Quick (smoke), or Regression. Default to diff-aware when on a branch with no URL.
  • Where the report goes — default is .qa-reports/; confirm only if the user wants it written elsewhere.

How this runs

Everything lives in the main conversation by default. The interactive spine demands it: auth ceremonies (2FA/OTP prompts, CAPTCHA hand-offs), cookie-import and sign-in confirmations, and any clarifying AskUserQuestion (which port, which URL, scope) all need a real-time answer from the user. The browser behind $B is a single stateful session and every screenshot is a file write — so all browser driving, screenshots, and report writes stay in the main loop too.

Two parts are pure, non-interactive compute: diff-impact mapping (map each changed file to the pages/routes/states it affects) and source-code review mode (read each affected page's source, trace data flow, build the State Coverage Check and manual test cases). Do both inline unless the user opts into parallelism — they are the only parts eligible for fan-out, and only after an explicit yes. Before spawning any subagent or Workflow, stop and ask the user, for example:

This branch touches [N files]. I can map the diff impact / review the source inline here (cheaper, slower), or fan out [M] parallel subagents (faster, more tokens). Which do you want?

Spawn subagents/workflows only after an explicit yes. If the user declines or doesn't answer, do the whole job inline.

If the user does opt in, respect the hard constraint: a Workflow agent() runs in the background and cannot call AskUserQuestion, drive the browser, or write files. So even then, every auth ceremony, OTP/CAPTCHA hand-off, clarifying question, $B action, screenshot, and report/baseline write stays in the main loop — the workflow only returns headless analysis. Set model: 'opus' on every agent() call, and pass data through args (Workflow scripts have no Date.now/Math.random/filesystem access; timestamps and the qa-report-{domain}-{YYYY-MM-DD}.md filename are computed in the main loop).

Arguments

Read $ARGUMENTS and route on what it contains:

  • A target URL (http://localhost:3000, https://staging.example.com) → run Full mode against it.
  • A PR number / commit ref / branch / feature name → scope to that diff, run diff-aware mode, and use the label in the report.
  • A file path or scope hint ("Focus on the billing page") → narrow scope accordingly.
  • A mode flag — --quick → Quick mode; --regression baseline.json → Regression mode; "report only" / "don't fix" → confirm the report-only posture (which is always on for this skill).
  • Empty while on a feature branch → default to diff-aware mode (the common case).

Setup

Parse the request for these parameters:

ParameterDefaultOverride example
Target URLauto-detect or requiredhttp://localhost:3000, https://staging.example.com
Modefull (URL given) / diff-aware (feature branch)--quick, --regression baseline.json
Output dir.qa-reports/ (confirm if the user prefers elsewhere)Output to /tmp/qa
Scopefull app or diff-scopedFocus on the billing page
AuthNoneSign in to user@example.com, Import cookies from cookies.json

No URL and on a feature branch? Enter diff-aware mode automatically. This is the usual situation — code just shipped on a branch and the user wants a read-only verification pass.

Browser automation tool ($B):

This skill drives a browser through a CLI helper referenced as $B (commands like $B goto, $B snapshot, $B console). Detect an available one in this order:

command -v browse >/dev/null 2>&1 && B=browse
[ -z "$B" ] && [ -x "./scripts/browse" ] && B="./scripts/browse"
[ -z "$B" ] && [ -x "./bin/qa" ] && B="./bin/qa"

If no browser CLI is available, fall back to source-code review mode: read component source, trace data flow, predict behavior, and document manual test cases. You still never edit files — the user runs those cases themselves.

Detect CDP mode (when $B is attached to a real browser):

$B status 2>/dev/null | grep -q "Mode: cdp" && echo "CDP_MODE=true" || echo "CDP_MODE=false"

If CDP_MODE=true: skip cookie-import prompts (the real browser already carries cookies), skip user-agent overrides, skip headless-detection workarounds.

Create output directories:

mkdir -p .qa-reports/screenshots

Step 0: Detect base branch

  1. gh pr view --json baseRefName -q .baseRefName — use it if it succeeds
  2. gh repo view --json defaultBranchRef -q .defaultBranchRef.name — use it if it succeeds
  3. Fallback: git symbolic-ref refs/remotes/origin/HEAD 2>/dev/null | sed 's|refs/remotes/origin/||'
  4. Final fallback: main — but if none of the above resolved and the base is genuinely ambiguous, confirm the branch to diff against rather than silently assuming main.

Test Plan Context

Before dropping to raw git-diff heuristics, look for a richer test plan to scope against:

  1. Conversation context — if a prior /plan-review or /scope-review in this session produced a test plan, use it.
  2. Git diff analysis — fall back to this when no test plan exists.

Modes

Diff-aware (automatic on a feature branch with no URL)

The primary mode for developers verifying their own work:

  1. Read the branch diff:
    git diff <base>...HEAD --name-only
    git log <base>..HEAD --oneline
  2. Map changed files to affected pages/routes. Do this inline by default. (On a large branch, you may offer to parallelize it — see How this runs — and only fan out via Dynamic Workflow: Diff-Impact Mapping if the user agrees.) For each changed file, classify:

    • Controller/route files → the URL paths they serve
    • View/template/component files → the pages that render them
    • Model/service files → the pages that use them (check controllers that reference them)
    • CSS/style files → the pages that include them
    • API endpoints → hit them directly with $B js "await fetch('/api/...')"
    • Static pages (markdown, HTML) → navigate directly

    No obvious pages/routes surface? Don't skip browser testing. Drop to Quick mode — load the homepage, follow the top 5 nav targets, watch the console. Backend/config/infra changes still alter app behavior, so always verify something.

  3. Find the running app — probe common local dev ports:
    $B goto http://localhost:3000 2>/dev/null && echo "Found app on :3000" || \
    $B goto http://localhost:4000 2>/dev/null && echo "Found app on :4000" || \
    $B goto http://localhost:8080 2>/dev/null && echo "Found app on :8080"
    If none respond, check the PR description or environment for a staging URL. Otherwise ask the user.
  4. Test each affected page/route (main loop — the browser is stateful):
    • Navigate to it
    • Screenshot it
    • Check the console for errors
    • If the change is interactive (forms, buttons, flows), walk it end-to-end
    • Use snapshot -D before and after actions to confirm the effect
  5. Cross-reference commit messages and the PR description to understand intent — what is the change supposed to do? Watch whether it actually does that.
  6. Check TODOS.md (if present) for known bugs tied to the changed files. Note any matches in the report.
  7. Report findings scoped to the branch:
    • "Changes tested: N pages/routes affected by this branch"
    • For each: does it work? With screenshot evidence.
    • Any regressions on adjacent pages?

Full (default when a URL is provided)

Systematic exploration. Visit every reachable page. Document 5–10 well-evidenced issues. Produce a health score. Roughly 5–15 minutes depending on app size.

Quick (--quick)

A 30-second smoke test. Load the homepage plus the top 5 nav targets. Check: does the page load? Console errors? Broken links? Produce a health score. No detailed issue write-ups.

Regression (--regression <baseline>)

Run Full mode, then load baseline.json from a previous run. Diff them: which issues are fixed, which are new, what is the score delta? Append a regression section to the report.


Dynamic Workflow: Diff-Impact Mapping (opt-in)

This runs only if the user opted into parallelism (see How this runs); otherwise map the diff inline. Mapping each changed file to the pages/routes/states it touches is N independent read-and-reason passes, so on a large branch it parallelizes cleanly — one agent per changed file, then assemble the affected-page set in the main loop. Background agents can't drive the browser; this only produces the test-target list.

export const meta = {
  name: 'qa-report-diff-impact',
  description: 'Parallel diff-impact mapping — one agent per changed file maps it to affected pages/routes/states',
  phases: [{ title: 'Map' }],
}
const IMPACT_SCHEMA = {
  type: 'object',
  properties: {
    file: { type: 'string' },
    fileKind: { type: 'string' },                                  // controller|route|view|component|model|service|css|api|static|config|other
    affectedRoutes: { type: 'array', items: { type: 'string' } },  // URL paths to test
    affectedPages: { type: 'array', items: { type: 'string' } },   // page/screen names that render or use this file
    interactive: { type: 'boolean' },                              // does the change touch a form/button/flow to walk end-to-end?
    statesToCheck: { type: 'array', items: { type: 'string' } },   // loading|empty|error|success|partial states this change can affect
    intent: { type: 'string' },                                    // what this change is meant to do (from commit msgs / code)
    notes: { type: 'string' },
  },
  required: ['file', 'fileKind', 'affectedRoutes', 'affectedPages', 'interactive'],
}
const impacts = await parallel(args.files.map(f => () =>
  agent(
    `You are scoping a read-only QA pass on a feature branch. Do NOT edit anything, do NOT drive a browser — ` +
    `return analysis only.\n\n` +
    `CHANGED FILE: ${f}\n` +
    `BASE BRANCH: ${args.base}\n` +
    `COMMIT CONTEXT:\n${args.log}\n\n` +
    `Read this file and the controllers/components that reference it. Classify its kind, then map it to the ` +
    `concrete URL paths/routes a tester should visit and the page/screen names that render or use it. ` +
    `Controller/route → URL paths it serves; view/template/component → pages that render it; model/service → ` +
    `pages that use it (trace referencing controllers); CSS/style → pages that include it; API endpoint → the ` +
    `endpoint path to hit directly; static page → navigate directly. Decide whether the change touches an ` +
    `interactive flow (form/button/async op) that must be walked end-to-end, list the UI states it can affect ` +
    `(loading/empty/error/success/partial), and state the change's intent from commit messages and code.`,
    { label: `map:${f}`, phase: 'Map', model: 'opus', schema: IMPACT_SCHEMA }
  )
))
return impacts.filter(Boolean)

Pass args: { files: ["<changed file>", ...], base: "<base branch>", log: "<git log <base>..HEAD --oneline output>" }. When it returns, union the affectedRoutes / affectedPages across all files into the test-target set, then drive the browser over each target in the main loop (Step 4). If nothing surfaces, fall back to Quick mode (homepage + top 5 nav targets). If the workflow fails or times out, map the diff inline.


Dynamic Workflow: Source-Code Review Mode (opt-in)

This runs only if the user opted into parallelism (see How this runs); otherwise analyze each page's source inline. When $B is unavailable, each affected page's source analysis is an independent read-and-reason pass, so on a large change it parallelizes — one agent per affected page, each reading the component source, tracing data flow (entry → validation → state → API → render), producing that page's State Coverage Check, and drafting manual test cases. Agents never edit files. Assemble the report in the main loop.

export const meta = {
  name: 'qa-report-source-review',
  description: 'Parallel source-code QA — one agent per affected page traces data flow and produces State Coverage + manual test cases',
  phases: [{ title: 'Analyze' }],
}
const PAGE_SCHEMA = {
  type: 'object',
  properties: {
    page: { type: 'string' },
    files: { type: 'array', items: { type: 'string' } },
    stateCoverage: {
      type: 'array',
      items: {
        type: 'object',
        properties: {
          feature: { type: 'string' },
          loading: { type: 'string' },   // HANDLED | MISSING | PARTIAL
          empty: { type: 'string' },
          error: { type: 'string' },
          success: { type: 'string' },
          partial: { type: 'string' },
        },
        required: ['feature', 'loading', 'empty', 'error', 'success', 'partial'],
      },
    },
    issues: {
      type: 'array',
      items: {
        type: 'object',
        properties: {
          title: { type: 'string' },
          severity: { type: 'string' },   // Critical | High | Medium | Low
          category: { type: 'string' },   // Functional | Visual | UX | Performance | Data Safety | A11y
          file: { type: 'string' },       // path/to/file.ts:LINE
          description: { type: 'string' },
          impact: { type: 'string' },
          evidence: { type: 'string' },   // code snippet showing the bug
        },
        required: ['title', 'severity', 'category', 'description', 'impact'],
      },
    },
    manualTestCases: {
      type: 'array',
      items: {
        type: 'object',
        properties: {
          id: { type: 'string' },           // TC-NNN
          precondition: { type: 'string' },
          steps: { type: 'array', items: { type: 'string' } },
          expected: { type: 'string' },
          verify: { type: 'string' },
          risk: { type: 'string' },
        },
        required: ['id', 'steps', 'expected', 'verify', 'risk'],
      },
    },
  },
  required: ['page', 'files', 'stateCoverage', 'issues'],
}
const pages = await parallel(args.pages.map(p => () =>
  agent(
    `You are doing a READ-ONLY source-code QA pass on one page. Do NOT edit any file — read and describe only.\n\n` +
    `PAGE: ${p.name}\nSOURCE FILES: ${p.files.join(', ')}\n\n` +
    `Read the component source and trace data flow end-to-end (entry → validation → state → API → render). ` +
    `For each UI feature on this page, produce the State Coverage Check: mark LOADING / EMPTY / ERROR / SUCCESS / ` +
    `PARTIAL as HANDLED (code explicitly handles this state with appropriate UI), MISSING (user would see blank/` +
    `broken UI), or PARTIAL (handled but with issues, e.g. spinner shows but never clears). Then document each bug ` +
    `you find with file path + line number + a code snippet + impact (NO suggested fix — report only), and draft ` +
    `manual test cases (precondition, steps, expected, verify, risk) for anything needing human verification.`,
    { label: `page:${p.name}`, phase: 'Analyze', model: 'opus', schema: PAGE_SCHEMA }
  )
))
return pages.filter(Boolean)

Pass args: { pages: [{ name, files: ["path/to/component.tsx", ...] }, ...] }. When it returns, fold each page's State Coverage Check, issues, and manual test cases into the report (main loop), assigning QA-NNN ids in order and writing each issue in the standard format below. If the workflow fails or times out, run the source-code analysis inline.


Workflow

Phase 1: Initialize

  1. Find the browser tool $B (or fall back to source-code review mode)
  2. Create the output directories
  3. Start a timer for duration tracking
  4. Read the project's CLAUDE.md and any project memory files for conventions

Phase 2: Authenticate (if needed)

When the user supplied credentials:

$B goto <login-url>
$B snapshot -i                    # find the login form
$B fill @e3 "user@example.com"
$B fill @e4 "[REDACTED]"          # NEVER put real passwords in the report
$B click @e5                       # submit
$B snapshot -D                     # confirm login succeeded

When the user supplied a cookie file:

$B cookie-import cookies.json
$B goto <target-url>

2FA/OTP required: ask the user for the code and wait.

CAPTCHA in the way: tell the user, "Please complete the CAPTCHA in the browser, then tell me to continue."

Phase 3: Orient

Build a map of the app:

$B goto <target-url>
$B snapshot -i -a -o ".qa-reports/screenshots/initial.png"
$B links                          # map navigation structure
$B console --errors               # any errors on landing?

Detect the framework (record it in report metadata):

  • __next in HTML or _next/data requests → Next.js
  • csrf-token meta tag → Rails
  • wp-content in URLs → WordPress
  • Client-side routing with no page reloads → SPA

For SPAs: links may return little because routing is client-side. Use snapshot -i to locate nav elements (buttons, menu items) instead.

Phase 4: Explore

Visit pages systematically. At each one:

$B goto <page-url>
$B snapshot -i -a -o ".qa-reports/screenshots/page-name.png"
$B console --errors

Then run the per-page checklist:

  1. Visual scan — read the annotated screenshot for layout problems
  2. Interactive elements — click buttons, links, controls. Do they respond?
  3. Forms — fill and submit. Try empty, invalid, and edge-case inputs
  4. Navigation — check every path in and out
  5. States — empty, loading, error, overflow
  6. Console — any new JS errors after interactions?
  7. Responsiveness — check a mobile viewport when relevant:
    $B viewport 375x812
    $B screenshot ".qa-reports/screenshots/page-mobile.png"
    $B viewport 1280x720

Depth judgment: spend more time on core surfaces (homepage, dashboard, checkout, search) and less on secondary pages (about, terms, privacy).

Quick mode: visit only the homepage + top 5 nav targets. Skip the checklist — just confirm: loads? Console errors? Visibly broken links?

When $B is unavailable (source-code fallback): for each affected page, read the component source, trace data flow (entry → validation → state → API → render), and document manual test cases per the Phase 5 template. Also produce the State Coverage Check below. By default do this inline; only fan it out via Dynamic Workflow: Source-Code Review Mode if the user opted into parallelism. Never edit files — read and describe only.

State Coverage Check (source-code mode)

For each UI feature on the affected page, confirm the code handles:

FEATURE              | LOADING | EMPTY | ERROR | SUCCESS | PARTIAL
---------------------|---------|-------|-------|---------|--------
[feature name]       | ?       | ?     | ?     | ?       | ?

Mark each "?":

  • HANDLED: code explicitly handles this state with appropriate UI
  • MISSING: code does not handle this state (user would see blank/broken UI)
  • PARTIAL: code handles it but with issues (e.g. spinner shows but never clears)

Phase 5: Document

Write each issue the moment you find it — don't batch.

Two evidence tiers:

Interactive bugs (broken flows, dead buttons, form failures):

  1. Screenshot before the action
  2. Perform the action
  3. Screenshot the result
  4. Use snapshot -D to show what changed
  5. Write repro steps that reference the screenshots
$B screenshot ".qa-reports/screenshots/issue-001-step-1.png"
$B click @e5
$B screenshot ".qa-reports/screenshots/issue-001-result.png"
$B snapshot -D

Static bugs (typos, layout issues, missing images):

  1. One annotated screenshot showing the problem
  2. Describe what's wrong

Source-code-mode bugs: file path + line number + a code snippet showing the bug + an impact description. No edits.

Write each issue to the report immediately, in this format:

### QA-NNN: [Title]

**Severity:** Critical / High / Medium / Low
**Category:** Functional / Visual / UX / Performance / Data Safety / A11y
**Page:** [URL path]
**File:** `path/to/file.ts:LINE` (if code-level)

**Description:** What is wrong
**Impact:** What the user experiences
**Repro steps:**
  1. Navigate to [page]
  2. Do [action]
  3. Observe [problem]

**Expected:** [correct behavior]
**Actual:** [broken behavior]

**Evidence:** [screenshot paths or code snippet]

Note: do NOT add a "Suggested fix" section. This skill reports only — /qa and /qa-fix own the fixes.

Manual test cases (for issues that need human verification):

MANUAL TEST CASE: TC-001
========================
Page: [URL]
Precondition: [state required]
Steps:
  1. [step]
  2. [step]
Expected: [behavior]
Verify: [observation]
Risk: [what could go wrong]

Phase 6: Wrap Up

  1. Compute the health score with the rubric below
  2. Write "Top 3 Things to Fix" — the three highest-severity issues (description only, no code)
  3. Console health summary — aggregate every console error seen across pages
  4. Update the severity counts in the summary table
  5. Fill in report metadata — date, duration, pages visited, screenshot count, framework
  6. Save a baseline — write .qa-reports/baseline.json:
    {
      "date": "YYYY-MM-DD",
      "url": "<target>",
      "healthScore": N,
      "issues": [{ "id": "QA-001", "title": "...", "severity": "...", "category": "..." }],
      "categoryScores": { "console": N, "links": N, ... }
    }

Regression mode: after writing the baseline, load the previous baseline and compare:

  • Health-score delta
  • Issues fixed (in the baseline but not current)
  • New issues (in current but not the baseline)
  • Append a regression section to the report

Write the final report to .qa-reports/qa-report-{domain}-{YYYY-MM-DD}.md (or the output location the user confirmed up front — see Gather context first):

# QA Report (Report-Only) — [branch name]

**Date:** YYYY-MM-DD
**Branch:** [branch]
**Base:** [base branch]
**Mode:** Diff-aware / Full / Quick / Regression
**Duration:** Nm
**Framework:** [detected framework]
**Type:** Report-only (no fixes applied)

---

## Summary

| Metric | Value |
|--------|-------|
| Pages/routes tested | N |
| Issues found | N (X critical, Y high, Z medium, W low) |
| Health score | X/100 |

## Top 3 Things to Fix

1. **[QA-NNN]** [title] — [one-line impact]
2. **[QA-NNN]** [title] — [one-line impact]
3. **[QA-NNN]** [title] — [one-line impact]

## Console Health

[Aggregated console errors across all pages]

## Manual Test Cases

[List of test cases for manual verification]

## All Issues

[Full issue details with evidence]

## Health Score Breakdown

| Category | Score | Issues |
|----------|-------|--------|
| Console | X/100 | N issues |
| Links | X/100 | N issues |
| Visual | X/100 | N issues |
| Functional | X/100 | N issues |
| UX | X/100 | N issues |
| Performance | X/100 | N issues |
| Content | X/100 | N issues |
| Accessibility | X/100 | N issues |

## Ship Readiness

| Check | Status |
|-------|--------|
| Critical issues | N |
| High issues | N |
| Health score | X/100 |
| **Verdict** | SHIP / SHIP WITH CAUTION / DO NOT SHIP |

**Verdict rules:**
- 0 critical + 0 high → SHIP
- 0 critical + some high → SHIP WITH CAUTION
- Any critical → DO NOT SHIP

---

_Report-only mode. No fixes applied. Run /qa or /qa-fix to fix issues._

If .qa-reports/qa-report-*.md already exists for today, append the new report below the existing content.


Health Score Rubric

Compute each category score (0–100), then take the weighted average.

Console (weight: 15%)

  • 0 errors → 100
  • 1–3 errors → 70
  • 4–10 errors → 40
  • 10+ errors → 10

Links (weight: 10%)

  • 0 broken → 100
  • Each broken link → −15 (minimum 0)

Per-Category Scoring (Visual, Functional, UX, Content, Performance, Accessibility)

Each category starts at 100. Deduct per finding:

  • Critical issue → −25
  • High issue → −15
  • Medium issue → −8
  • Low issue → −3

Minimum 0 per category.

Weights

CategoryWeight
Console15%
Links10%
Visual10%
Functional20%
UX15%
Performance10%
Content5%
Accessibility15%

Final Score

score = Σ (category_score × weight)


Framework-Specific Guidance

Next.js

  • Watch the console for hydration errors (Hydration failed, Text content did not match)
  • Watch _next/data requests in the network — 404s mean broken data fetching
  • Test client-side navigation (click links, don't just goto) — this catches routing issues
  • Check for CLS (Cumulative Layout Shift) on pages with dynamic content
  • useRouter null safety (query params undefined on first render)
  • Missing key props in dynamic lists

Rails

  • N+1 query warnings in the console (development mode)
  • CSRF token presence in forms
  • Turbo/Stimulus integration — do page transitions feel smooth?
  • Flash messages appearing and dismissing correctly

WordPress

  • Plugin conflicts (JS errors from different plugins)
  • Admin-bar visibility for logged-in users
  • REST API endpoints (/wp-json/)
  • Mixed-content warnings

General SPA (React, Vue, Angular)

  • Use snapshot -i for navigation — links misses client-side routes
  • Stale state (navigate away and back — does data refresh?)
  • Browser back/forward — does the app handle history correctly?
  • Memory leaks (watch the console after extended use)

React + Redux

  • Selectors handle initial state (before data loads)
  • Dispatch calls use correct action types
  • Async mutations cover all paths (loading, success, error)
  • useEffect dependency arrays complete
  • Optional-chaining gaps on Redux state

API layer (axios/fetch)

  • Error responses checked (status codes, error_code fields)
  • Loading states set and cleared on every path (including error)
  • Race conditions (component unmounts during an API call)
  • Missing try/catch around API calls

i18n

  • Hardcoded strings that should use t() or a translation function
  • Missing translation keys
  • Locale fallback behavior

Important Rules

  1. Never fix anything. Report only. Do not edit source files.
  2. Never suggest fixes in the report. Describe what is wrong, not how to fix it. /qa and /qa-fix own the fixes.
  3. Never make commits. No git commit, no git add, no git revert. Reads only.
  4. Repro is everything. Every issue needs at least one screenshot (or file:line in source-code mode). No exceptions.
  5. Verify before documenting. Retry the issue once to confirm it reproduces — not a fluke.
  6. Never include credentials. Write [REDACTED] for passwords in repro steps.
  7. Write incrementally. Append each issue to the report as you find it. Don't batch.
  8. Check the console after every interaction. JS errors that don't surface visually are still bugs.
  9. Test like a user. Use realistic data. Walk complete workflows end-to-end.
  10. Depth over breadth. 5–10 well-documented issues with evidence beat 20 vague descriptions.
  11. Never delete output files. Screenshots and reports accumulate on purpose.
  12. Show screenshots to the user. After every $B screenshot, $B snapshot -a -o, or $B responsive command, Read the output file(s) so the user sees them inline. For responsive (3 files), Read all three. Otherwise the screenshots are invisible to the user.
  13. Justify severity. Explain why an issue is critical vs high vs medium.
  14. Always produce the report. Even when no issues are found.
  15. The ship-readiness verdict is mandatory. Never finish without one.
  16. Never refuse the browser. Invoking qa-report is a request for browser-based testing. Don't offer evals, unit tests, or alternatives as a substitute. If $B is unavailable, fall back to source-code review mode (still read-only).
  17. Default to inline; keep interactive and browser work in the main loop. Run diff-impact mapping and source-code review inline by default; fan them out only after the user opts in (see How this runs). Even then, never put an auth ceremony, OTP/CAPTCHA hand-off, $B action, screenshot, or any file write inside a Workflow agent() — background agents can't talk to the user, drive the browser, or write files.

Output Structure

.qa-reports/
├── qa-report-{domain}-{YYYY-MM-DD}.md    # Structured report
├── screenshots/
│   ├── initial.png                        # Landing page annotated screenshot
│   ├── issue-001-step-1.png               # Per-issue evidence
│   ├── issue-001-result.png
│   └── ...
└── baseline.json                          # For regression mode

Report filenames carry the domain and date: qa-report-myapp-com-2026-03-12.md


Handoff

When the issues need fixing:

  • /qa — full test → fix → verify loop with atomic commits, regression tests, ship verdict
  • /qa-fix — targeted source-level analysis and fixes for specific bugs

This skill stops at the report. Nothing else.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.