Manual Test

Drives manual / exploratory QA against a web app. Reads the test context (from an issue-tracker ticket or directly from the user), executes it by piloting a real browser through a throwaway Playwright script, has the evidence independently reviewed, and reports pass/fail. Does not commit any test files — every script written here is throwaway and lives in a git-ignored scratch root.

Pairs with create-test-plan-scenarios (produces the checklist) and file-bug-ticket (files the bugs this run finds).

Model tiers used below

Tiers are capability levels, not vendor names — map them to what your harness offers:

deep+ means the top tier and the highest reasoning effort the harness exposes.

Step 0 — Triage & delegation (mandatory)

This skill is a router: it does not execute the test in the main conversation. A browser run produces a lot of tool output (browser, network, console) that should not fill the conversation, and the review only works if the reviewer did not watch the run.

Stays in the main conversation (orchestrator):

• Decide the mode, load the ticket, derive the plan (Steps 1–2).
• Ask the user for anything you cannot access (external docs, designs, pasted content).
• Confirm the environment. Default is staging; production only with an explicit instruction in this turn.
• Confirm how to seed any state the standard fixture cannot reach.
• List the known bugs that cross the plan, so the executor does not re-investigate them.
• Read the project's knowledge bank, when it has one, and route each entry to the brief or the reviewer (Step 1).
• After the executor returns: dispatch the independent reviewer (Step 6) and, only with the user's approval, the tracker writes (Step 7) and the knowledge-bank update (Step 8).

Three distinct roles, in this order: executor (Steps 3–5) → reviewer (Step 6) → tracker writer (Step 7, via file-bug-ticket). The knowledge-bank update (Step 8) stays with the orchestrator. The reviewer is never a fork of the executor or of this conversation: it must reach the evidence without inheriting the conclusions of whoever produced it. Pass context through the prompt, not through inheritance.

How to dispatch, in any harness

Use the first option your harness supports:

1. Packaged agents — if qa-test-executor and qa-evidence-reviewer are installed (Claude Code plugin, or the OpenCode/Codex adapters in this repo), dispatch them by name (the plugin namespaces them, e.g. qa-flow:qa-test-executor). They carry the role prompt with its references inlined, so do not point them at roles/ — they spend the first turns searching for it. In Claude Code and OpenCode they also get a best-effort permission guard (reviewer read-only; no tracker writes or commits); in Codex the boundary is instruction-only.
2. Generic subagent — spawn a fresh-context subagent and make its first instruction: "Read <absolute path>/roles/test-executor.md (or roles/evidence-reviewer.md) and follow it" — resolve the path next to this SKILL.md yourself; the subagent does not know where it is. Repeat the "never write to the tracker" rule in the brief, since nothing enforces it.
3. No subagents at all — play the roles yourself, sequentially. For the review, start a clean pass: re-open the evidence from disk, ignore your earlier conclusions, and state in the report that the review was not independent.

The executor brief

A thin prompt ("run these scenarios") is what produces a fragile report. Carry all six sections:

[Generic subagent only: Read <absolute path>/roles/test-executor.md and follow it.]
Environment: <staging|prod>. User/fixture: <which>.
Recon template: <absolute path to this skill>/assets/recon.template.spec.ts — copy it, fill in
OUT / ENTRIES / overlay only. [Or: "Recon may be skipped — <reason>": page objects cover the whole
flow / fresh knowledge-bank locators for every target: … / single assertion, locator: … /
first-visit state (or: "run recon as <second fixture user>")]

## Ticket and ACs
<ACs verbatim — do not paraphrase; the executor compares this text with the screen>

## Scope
In scope: <...>. Out of scope: <...> — and WHY (e.g. data for X does not exist yet).
If something out of scope fails, record it as an observation, not a ticket failure.

## Plan (already resolved — run in this order, do not re-derive)
<numbered list, with what to observe in each item>

## Known bugs — do NOT investigate, do NOT file
<ID: one-line repro> → if crossed, label BLOCKED and move on.

## App/environment gotchas
<bot protection, overlays to dismiss, URL-based seeding, selectors,
 baseline console noise, short timeouts, network capture by blocklist,
 entry URLs for recon — or why recon can be skipped>

## Rules
Throwaway files in _local/. Do not commit. Do not write to the tracker.
Recon first, read-only; it shows how to reach elements, never what to expect.
Evidence must prove what it claims — a failed click is not a new state.
Do not ask anything; if stuck, return what blocked you, what passed, and the evidence collected.

Tiers per role

A second pass after the review inherits the executor's tier, never less — it attacks exactly what was hard the first time. If a subagent comes back blocked, resolve it with the user and redispatch one tier up instead of taking the work over in the main conversation.

Do not delegate guided mode (the user performs the steps and reports what they saw) — that is a conversation and runs inline. If the executor discovers mid-run that guided mode is needed, it returns and the orchestrator takes over.

Two modes

• Ticket mode — the user passes a ticket id (TICKET-1234, a bare number, or a full URL). Load the ticket, derive the plan from the description / ACs / existing checklist, run each item in order.
• Ad-hoc mode — the user pastes a single scenario in plain language. Execute that one scenario only, no ticket lookup.

If the user gives both, ad-hoc wins; ask whether to still pull the ticket for context.

Configuration (fill these in for your setup)

• App under test — staging and production URLs. Default to staging; never hit production without an explicit instruction in this turn.
• E2E repo — run from your Playwright project so fixtures and page objects are reusable.
• Auth fixtures — your stored storageState file(s), e.g. playwright/.auth/user.json.
• Page-object helpers — prefer them over raw selectors when one already covers the flow.
• Knowledge root — where the project keeps its knowledge bank (default .qa-flow/knowledge/, optional). Curated, not throwaway: see references/knowledge-bank.md.
• Scratch root — one git-ignored folder for everything throwaway (default _local/, with scripts/, recon/<slug>/, evidence/<slug>/, logs/). Never inside the committed suite or the repo root.
• Tracker — TRACKER_API_BASE and TRACKER_TOKEN, read at runtime from env or a git-ignored .env.local. Never inline, never commit, never print.

Auth & secrets

• Tracker token: from env / local secrets file. On 401, ask the user for a fresh token, update the local file, retry once.
• App credentials: in a git-ignored .env; your global/auth setup generates the storageState fixtures. If a fixture is missing or older than the session lifetime, regenerate it or ask the user.
• Bot protection / WAF: if your app sits behind one, the automated session may need a specific userAgent or header — configure it in the throwaway config, sourced from env. A silent challenge page looks like "login failed" but isn't.

Workflow

1. Resolve mode & gather context

Ticket mode:

1. Normalize the id and fetch the ticket via the tracker API.
2. Pull: title, description, type, custom fields, comments, related-ticket links, existing checklist, and the linked branches / pull requests.
   • PRs say what was actually delivered. A follow-up PR named fix: … points straight at what needs extra verification — e.g. a fix: … PR that touches the mobile layout turns "test on mobile" into a mandatory scenario.
   • PR merged but the AC is not observable in the app → probably a pending deploy. Confirm before failing it.
3. For each link in description / comments (docs, designs, wikis), try to fetch it. If it needs auth or returns junk, ask the user to paste the relevant content — be specific about what you need — and wait. Never invent expectations to fill the gap.
4. Read every comment — product owners clarify copy, IDs and edge cases there. Comments override the description.

5. List the known bugs that cross this ticket, BEFORE running. A run that rediscovers a tracked bug wastes the turn and pollutes the report with a false-new finding. Look at:

   • the ticket's links (what already blocks it);
   • sibling bugs in the same epic/project;
   • a tracker text search for the feature name restricted to bugs;
   • the knowledge bank's known-bug entries — pointers only: confirm each one's state in the tracker before listing it;
   • your project notes / persistent memory — environment bugs usually live there.

   For each, write into the plan: "if crossed, label BLOCKED — <id> known, do not investigate, do not file".

6. Confirm with the user/PO what has no data or backend yet. Front-end delivered before its backend is common and changes the reading of half the ACs — those become BLOCKED-BY-DEPENDENCY, not FAIL. Ask before running, not after failing.

Both modes — read the knowledge bank (when the project has one). Run node <this skill's dir>/scripts/check-knowledge.mjs <knowledge root> --json first: it flags stale entries and anything that should not be there. Then read the files for the areas the run touches and route each entry as references/knowledge-bank.md says — gotcha, locator and data into the executor brief; known-bug into Known bugs after checking the tracker (unreachable tracker → leave it out and say so); fresh intent only to the reviewer, as "verify against <ticket>", never to the executor and never as a verdict; a stale intent is a question for the user first. Entries the checker flags are not routed — propose the fix. Stale entries go in marked "unverified since <date>". Fresh locator entries let you skip recon only when they cover every AC target and navigation step — list them in the brief (the other reasons to skip it are in references/recon.md: page objects, a single assertion, first-visit state — and the brief always names the reason).

Ad-hoc mode: take the scenario verbatim. If it references external context ("as in ticket X", "like in design Y"), follow the same fetch-or-ask loop.

2. Draft an executable plan

Numbered atomic, observable steps, each with a precondition (which user, which screen), an action (one interaction) and an expected result (verbatim copy from the spec when given). If the ticket already has a QA checklist, use it as the source of truth — don't re-derive.

Classify each AC before running, and write the class into the plan:

Coverage — sweep the axes, not just the happy path. A first pass tends to exercise one path per AC and declare PASS; that is how layout and state bugs slip through. Before closing the plan, ask:

• Alternative paths to the same AC. "Clicking X or Y" → test both. Two entry points to the same state → test both; the same click can work from one context and be swallowed from the other.
• State, not just render. Round-trip with filters filled in (not on an empty screen), refresh, browser Back/Forward, deep link, and what persists server-side.
• Breakpoints, when the change touches layout — mandatory when the ticket has a layout PR. Measure at ≥4 widths (e.g. 1600 / 1440 / 1280 / 1024 / mobile) and with a filter applied — a counter like "(1)" changes the bar's geometry. One viewport hides the bug.
• Account tiers and gating. If a list has gated and free items, exercise at least one of each. "Tested 2 of 7 types, both free" does not cover an AC that says "any".
• Mobile is another component, not the same one narrower. A control in the desktop bar may become a sheet on mobile — re-verify the AC inside it, not just the stacking.

Frame the scope with the user when the ticket is front-end without backend. Say what stays out and why, before running.

Show the plan before executing only when: the ticket is ambiguous and you made non-trivial assumptions; the plan exceeds ~10 steps; or the test is destructive (delete, payment, sending email). Otherwise proceed — the user invoked this skill to see the run, not to re-approve it.

3–5. Execute and report (executor role)

Dispatch the executor (Step 0) with the six-section brief. Its procedure is in roles/test-executor.md; the templates it uses are in references/playwright-templates.md, the read-only recon it runs before writing the spec in references/recon.md, its evidence rules in references/evidence-discipline.md, and the report it returns in references/report-format.md.

When it returns, check the machine-checkable JSON block first: every PASS cites evidence, no duplicate screenshots back different steps, every finding has a severity proposal, and every script_revisions entry is of a kind on the canonical list in references/recon.md (never an AC target or an expected value).

Fallbacks:

• No Playwright path possible (mobile-only flow, third-party iframe blocking automation, payment sandbox): guided mode, inline — print the steps one by one, ask the user to perform each, wait for their observation, compare, record. Save their screenshots to the evidence folder.
• A state you cannot reach with the standard fixture (account flag, paid plan, seeded data): pause and ask the user how to seed it. Don't fabricate one.

6. Independent review (mandatory before any write)

Nothing goes to the tracker without passing a reviewer. Dispatch it per Step 0 (fresh context, never a fork). Its procedure is in roles/evidence-reviewer.md.

Its prompt carries, in writing: the ACs verbatim; the agreed scope and why; the known bugs that do not count as findings; the knowledge bank's fresh intent entries for this area (and stale ones the user confirmed), each as "verify against <ticket>: <summary>" with its last_verified; what was executed and the result item by item; the findings with their proposed classification; and the paths to the evidence, recon output, logs and scripts, plus the script_revisions list — the reviewer must be able to open the screenshots and read the scripts, not just trust the report. It returns four sections: missing scenarios, fragile conclusions, findings to promote/demote (with severity), and sign-off.

After the review, the orchestrator decides: run a second pass on the gaps, or move to filing. Take the decision to the user when the second pass costs real time — and say whether any possible outcome of it could block the ticket. A pass that only hunts failure paths, fault injection or rare timing is informational: the ticket moves on with whatever it finds.

Every finding carries blocking or non-blocking (Low) per the severity gate. The default is not to block, and promoting a finding to blocking is the user's call. Before accepting any finding — the executor's or the reviewer's — check it against product intent (same file).

Save the reviewed result to <scratch root>/reports/<slug>.json as soon as the review is in: the executor's JSON block plus, at the top level, "reviewer_verdict": "PASS | FAIL | PARTIAL", "accepted_steps": [ids the reviewer accepted as PASS], "confirmed_findings": [titles] and "reviewed_at": "YYYY-MM-DD" — the reviewer's verdict is the one later skills trust. No approval needed — it is a throwaway file like the rest of the scratch root, written whether or not anything is filed — and it is what scan-regression-candidates and promote-to-regression read later.

7. Follow-ups: comment on the ticket and file the bugs

Only after Step 6, and with the user's approval. All in English. Full rules in references/filing-and-comments.md:

• One bug = one root cause; mirror a sibling bug's fields; link with relates to, never blocks; evidence attached and referenced by filename; verify everything with a GET.
• File bugs through the file-bug-ticket skill with fields already resolved and the body already written by you, in a file.
• A bug goes in as blocking only if the user's approval says so — "file the bug" approves the filing, not a promotion past the severity gate's default.
• One index comment on the parent, ≤ 800 characters of prose, with evidence embedded when it is the only artifact.
• Mark checklist items complete only for clean passes, after asking once.
• Do not move the ticket between columns — suggest; the user decides.

8. Knowledge-bank update (proposed, then approved)

After Step 6 — whether or not anything was filed in Step 7: ad-hoc runs and clean passes also teach something. Propose a short diff of the knowledge bank: new gotchas, locator entries only for steps the reviewer accepted as PASS, intent entries only with a source (ticket / PO comment), entries the run contradicted (update or remove) and entries it confirmed (bump last_verified — for an intent, only if its source was re-opened this round). Run the checker on a scratch copy of the proposal before showing it, and ask about it as its own question: a yes to Step 7 or silence is not a yes to the knowledge bank. Write only what was approved; anything that still needs changing after the write is shown again first. Rules and what never goes in: references/knowledge-bank.md. The executor and the reviewer never edit the knowledge bank.

If the project has no knowledge bank, record what the next session would have to rediscover in your project notes / memory instead — or offer to start one.

Rules

• Faithful execution. Exact URLs, copy, IDs and account types from the ticket/spec. If something is missing, ask — do not improvise.
• No workarounds for app bugs. A real bug fails the test. No retries or try/catch to turn a red step green.
• Never commit the throwaway script. The scratch root is the only place it may exist.
• Do not delete anything until the user says the work is finished. Keep scripts, configs, logs and evidence for the whole session — the user may re-run, iterate or inspect. Never auto-clean.
• Staging by default. Production needs an explicit "run on prod" in this turn.
• One scenario at a time in ad-hoc mode. If the user pastes several, ask which first.
• All shared artifacts in English; conversation may stay in the user's language.
• Be explicit when you cannot access something. "I couldn't open design X — please send a screenshot of component Y" beats a guess.
• Text budget for the tracker: comment ≤ 800 characters of prose; bug body per file-bug-ticket (1000, or 500 for a simple front-end bug). Count before posting; if a real reason needs more, ask.
• Nothing reaches the tracker without the Step 6 review. The cost of a wrong finding there is credibility with the team.
• The knowledge bank informs, it never decides. An entry can tell the executor how to reach a page or the reviewer what to re-check; it never sets an expected result, skips a step or demotes a finding. No credentials, personal data or URLs with query strings in it, ever.
• Evidence must prove the claim. In doubt, INCONCLUSIVE.
• Before filing a bug, look for the sentence that says it is the product working.
• A known bug is not a finding. Label, reload, move on.
• Never leave a browser window hanging. Short action timeouts; close the context if a run dies.
• A narrow finding does not block a feature — apply the severity gate.
• Evidence always ships with the result. A results comment without its screenshots is incomplete.

Errors

• Tracker 401 → token rotated. Ask the user, update the local secrets file, retry once.
• Tracker 404 → wrong id. Print the id used and stop.
• Playwright login failure (storageState expired) → regenerate the auth fixture or ask the user to refresh credentials. Behind bot protection? Confirm the userAgent/header is configured.
• Selector not found / not visible within the action timeout → for an AC target, a test finding, not a tooling error: report it as the step's observation, with the recon snapshot if it shows the element was never on the page. For a navigation locator absent from the recon, a scripting error: fix it from the recon, log it in script_revisions, re-run (references/recon.md).
• External link unfetchable → ask the user for the content. Don't proceed on guesses.

Notes

• Read-heavy on the tracker API, write-light. Writes happen only after review and explicit confirmation.
• If the same scenario will be re-run often, suggest promoting it to a committed spec in your suite — as a suggestion only.