Quality Assurance

Overview

This skill executes verification scenarios against a known expected behaviour (spec, acceptance criteria, or task description) and produces an evidence-backed verdict. It covers browser UI flows, API verification, and regression checks — always starting from a spec anchor, never from the current implementation.

When to Use

Use this skill when:

• Verifying that a new feature works end to end
• Confirming a bug fix resolved the reported issue
• Running a regression check after any code change
• Executing a functional test plan for a feature
• Performing a smoke test before or after deployment

Do not use this skill when:

• Writing new tests or designing test coverage → use test-driven-development
• Isolating a bug's root cause → use debugging-and-error-recovery
• Doing a visual/design audit → use browser-testing-with-devtools

Core Process

Step 1 — Establish Expected Behaviour (mandatory)

Before executing anything:

1. Look for a spec, under the project's external output location (see references/external-output-paths.md) — specs/<feature-slug>/SPEC.md's Success Criteria section — or a legacy in-repo SPEC.md, task description, acceptance criteria, or issue description
2. If found — list the specific behaviours to confirm (one per scenario)
3. If not found — stop and ask: "What is the expected behaviour for [X]? I need a reference before I can verify."

Never infer expected behaviour from the current implementation. That makes verification circular — you'd confirm the code matches itself, not that it matches intent.

Step 2 — Plan Scenarios

If a Gherkin test-cases.md already exists for the feature, its scenarios are already concrete verification scenarios — execute those directly rather than re-deriving them from the feature spec. Otherwise, map expected behaviours to concrete verification scenarios:

Don't stop once one scenario type is covered. Checking that all 6 endpoints reject unauthenticated requests is a real result, but it's one type (error handling) — it says nothing about whether the happy path actually works, whether validation rejects bad input, or whether the specific fix/edge case this feature exists for behaves correctly. Every scenario type that genuinely applies to this feature needs at least one concrete check before the verdict is reported — not just whichever type was easiest or fastest to exercise.

Step 3 — Select Transport

Choose the right execution channel for each scenario. For full auth setup and Playwright MCP guidance, see transport-and-auth.md.

Every transport here means a real, running instance — not a mocked stand-in. This skill's whole value is proving the feature works against real dependencies (a real server process, a real database, a real authenticated user), not a client double standing in for one. A project's own test suite naming isn't proof of this: a suite called "e2e" can still run fully mocked, or can hit a real server but have an earlier guard/middleware reject the request before the code under test ever runs — verify the request actually reached and exercised the target logic, don't take the suite's name at face value.

If genuine live verification needs something not currently available — a real signed-in user's access token, a live database connection, an API key, test account credentials — don't silently substitute a mocked check, and don't rigidly block on the heaviest option either. Assembling full live access can genuinely be tedious; when it is, use AskUserQuestion to offer the real tradeoff instead of deciding unilaterally:

• Full live verification — real credentials/access, the most complete option, when it's not much friction to set up
• A minimal live check via curl/CLI — against the real running dev server (real process, real database) but a narrower surface, e.g. without a fully authenticated session if that's the specific friction point
• An automated test that boots/hits the real dev server and database — still genuinely live (real dependencies, not mocked), just run through the test framework rather than a manual browser/API session

All three are live in the sense this skill requires — none of them is a mocked substitute. Let the user pick the tradeoff; don't downgrade to mocks on your own, and don't insist on the full option when a lighter live check would do. If even the lightest live option isn't currently possible, say so explicitly in the report (Step 5) — "not verified live: requires X" is honest; a PASS verdict resting on mocks presented as equivalent is not.

Step 4 — Execute Scenarios

Drive each scenario to completion, against the real instance selected above:

• Use role+name locators for browser interactions, not positional refs
• Seed test state through the same channel as the test (browser state ≠ API session)
• Apply the two-strikes rule: if a step fails twice with the same error, stop and report — do not iterate

Step 5 — Produce Verification Report

## Verification Report

**Feature / Fix:** [Name or issue reference]
**Spec reference:** [Source of expected behaviour]

### Summary
**Overall verdict:** PASS | FAIL | PARTIAL

### Scenario Results

| Scenario | Expected | Observed | Verdict |
|---|---|---|---|
| [Name] | [Spec says] | [What happened] | PASS / FAIL |

### Issues Found
- **[Critical / High / Medium]** — [Description + evidence]

### Regressions
- [Any previously passing behaviour that now fails]

### Evidence
- [Screenshots, response bodies, console output]

Common Rationalizations

Red Flags

• Producing a PASS verdict without observable evidence
• Starting execution before identifying expected behaviour
• Skipping regression scenarios because "only one thing changed"
• Reporting a verdict having exercised only one scenario type (e.g., only that requests get rejected) when happy-path, validation, or fix-specific scenarios genuinely applied too
• A verdict resting on mocked/unit-level tests, or a suite merely labeled "e2e"/"integration", presented as if it were live verification
• Missing credentials or test access led to silently running a mocked check instead of offering the lighter live options first
• Full live verification assumed to be the only acceptable option, with no AskUserQuestion offering the curl/CLI or real-dev-server-test alternatives when the full setup was genuinely tedious
• Using API-seeded state for a browser test (separate sessions)
• Retrying a failing step more than twice without stopping to report

Verification

Exit criteria — all must be met before marking QA complete:

• Expected behaviour source is documented (spec file, issue link, or user confirmation)
• Every planned scenario has a recorded result (PASS / FAIL + evidence)
• Every scenario type that genuinely applies (happy path, edge cases, error handling, fix confirmation, regression) was actually exercised — not just one type
• Verification actually ran against a real, live instance (real server, real database, real authenticated user) — not a mocked client, and not just a suite that happens to be named "e2e"/"integration"
• If full live verification was genuinely tedious to set up, graduated live alternatives (curl/CLI against the real dev server, or an automated test hitting real dependencies) were offered via AskUserQuestion rather than downgrading straight to mocks
• If no live option was possible at all, that was stated explicitly in the report — never silently substituted with a mocked check reported as if equivalent
• Regression scenarios were run (at least the scenarios most likely affected by the change)
• Issues found are categorised by severity with reproduction steps
• Overall verdict (PASS / FAIL / PARTIAL) is stated explicitly