Orchestrating Elite Agent QA

When to use

• Multi-agent build/review/merge work where the quality bar is high.
• Shipping UI, interactive editors, or any surface where one build pass plus a polite review lets bugs through.
• Gating a merge on more than a build agent's "done".
• A slice needs a browser-driven visual gate, not just a code read.

When NOT to use

• Single-agent, low-stakes changes — overhead isn't worth it; orchestrating-agent-delegation alone covers the brief and tiering.
• Pure backend/logic changes with no rendered surface — the design-critic gate has nothing to check.
• You are the executing agent, not the orchestrator running the pipeline.

Overview

One build pass plus one agreeable review ships bugs. Quality comes from three independent forces stacked on every slice:

1. Adversarial diversity — several skeptical reviewers, each a different lens, prompted to refute.
2. An independent visual gate — an agent that drives the browser (not just reads code).
3. Realistic QA — a fresh, simple case exercised with real-mouse-like events, not the dense seed clicked by exact handle.

Gate the merge on all three; each catches what the others miss.

(The delegation mechanics under this pipeline live in orchestrating-agent-delegation; this skill is the UI-quality-gate specialization.)

The Per-Slice Pipeline (run in order)

1. Scope ONE slice → branch off main. Small, independently mergeable.
2. Design first (complex slices) — a planning agent reads the real code and designs the logic + exact file/function seams before any build. Skip for trivial slices.
3. Build — one agent implements on the branch. It must NOT commit. For UI that one agent is a single designer lane on opus owning the whole surface (every state, every mode step 6 names); never split a surface across builders — the seams show. Reviewers and the critic stay separate: they check, not shape.
4. Review — N parallel adversarial reviewers, each a distinct LENS (e.g. correctness · invariants · regression · headline behavior). Prompt them refute-first: "try to break it; report ONLY real, high-confidence issues with a concrete fix; empty if clean." Not a rubber stamp.
5. Fix — one agent applies the confirmed high/med findings (defer/note the lows).
6. Design-critic visual gate — a dedicated agent drives the browser (cache-bust the JS, screenshot each of the bar's modes: light, dark, increased contrast, forced colors; open menus/states) and challenges the render against the elite bar. Its high/med findings GATE the merge.
7. QA on a FRESH + REALISTIC setup — verify on a fresh simple case (2–3 named items), not only the complex seed, with realistic pointer sequences (pointer-capture, pointermove with buttons:1, dispatch at the location, not the exact data-* handle).
8. Merge only when build + review + critic + QA all pass. Update the SPEC/docs and plan in the same PR. QA issues get FIXED, never shipped.
9. Interaction slices: get the human's real-mouse confirmation — synthetic events can't exercise pointer-capture, hover, or thin hit-targets.

Orchestration Discipline

• Sequence slices that touch the same core files. Parallelize only truly independent work.
• A branch is not an isolated working tree. Parallel agents in the SAME checkout share one working directory: each needs a worktree.
• A green suite from a shared checkout is not evidence, and a red one is not attributable: other lanes' half-finished work is in it, even for read-only reviewers. Whole-suite claims come from a disposable worktree at trunk HEAD with only that agent's diff; every number names its tree.
• "Completed" ≠ done. A workflow can stop cleanly with the build finished but the review unrun. Verify the artifact (gate-check + a functional probe) before trusting any "done."
• Be resilient to flaky infra. If a workflow's parallel phase dies (overload, stall), don't just retry — take over with direct background agents (individual spawns survive when a fan-out panel doesn't) or finish inline. A workflow that produced its build can be resumed from cache; the dead review reruns live.
• Cache is the silent liar. Cache-bust the JS (fetch-reload + page reload) and confirm the new symbol loaded before believing a browser-QA result.
• Verify on the engine and device that ship. A headless engine, a responsive emulator or a narrowed desktop window is evidence about modules, not the product: it lacks the real compositor, browser chrome, scroll process and input. Every QA result names its engine and device; a phone surface needs a simulator or a device (ios-safari-surfaces).
• Prove every check can fail. Before believing a check, run a control both ways: a known-bad case must go red, a known-good case green (guarding-a-regression, proving-an-absence).

Common Mistakes

Copyable Prompt (hand to an orchestrator agent)

Per slice: branch off main. Non-trivial → first a design agent reads the real code and specifies the algorithm + exact file/function seams. A build agent (no commit). 3 parallel adversarial reviewers, distinct lenses (correctness · invariants/regression · the headline behavior), refute-first — only real, high-confidence issues with a concrete fix, empty if clean. A fix agent on confirmed high/med. A design-critic that drives the browser (cache-busted JS, every theme and mode, states exercised) gates the merge. QA it yourself on a fresh, simple setup with realistic pointer events, on the engine and device that ship. Merge only when all gates pass; update spec + plan in the same PR; fix QA issues, never ship them. Sequence slices that share files; verify "done" before trusting it; control-run every check; for interaction slices, get a human real-mouse confirmation.

Verification

1. Build, the N parallel refute-first reviewers, the design-critic pass, and fresh-case QA are each independently confirmed to have run — never inferred from a "completed" status alone.
2. The design-critic's screenshots come from cache-busted JS, in every mode step 6 names.
3. QA ran against a fresh, simple case with realistic pointer events — not only the dense seed clicked by exact handle.
4. For interaction slices, a human gave real-mouse confirmation before merge.
5. Every gating check has a recorded control run (known-bad red, known-good green), and every QA result names the engine and device it ran on.

Cross-references

• For the two claims a panel most often accepts unchecked: guarding-a-regression — a fix's regression test is evidence only once watched going red, and the panel is where that gets asked; and proving-an-absence — "no test covers this", "I could not reproduce it" and any bare count are absence claims, and an uncontrolled probe is how a false one survives review.
• For a run that outlives one sitting: session-heartbeat — this skill says which gates a slice must clear, not how a pipeline continues across a usage-limit reset or resumed session; pacing, the checkpoint, and re-firing a gate whose producer was killed live there.

Sources

• Practitioner-derived from the same #136 skill-unification build orchestrating-agent-delegation cites — this skill is that run's panel / design-critic / QA gate sequence.