qa-orchestrator
CLAUDE.md0 starsChanged 37 days agoMITView on GitHub
A Claude Code plugin (qa-orchestrator, marketplace aneskurtovic), not an application. It runs a multi-agent QA workflow for one Jira ticket inside a target repo. Nothing here is project-specific; a target opts in with a committed .qa/config.json (a filled-in example is config/examples/full-example.json). Docs. Only README.md and this file sit at the root; every other doc is in docs/ (the map is in README "Docs"). Keep the set small: a new doc needs a purpose no existing one covers. - docs/design.md: how…
The file
We split this CLAUDE.md into the rules it gives an agent
3 rules. About 0 are common, 0 are distinctive, and 3 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 12
Docs: Only README.md and this file sit at the root.
- Very specific: line 51
If you change the floor, change the CI matrix too.
- Very specific: line 106
Never call spawnSync directly in a script.
≈ “Common” is an estimate: at least 1 in 5 of the 26,980 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
What this repo is
A Claude Code plugin (qa-orchestrator, marketplace aneskurtovic), not an application. It runs a
multi-agent QA workflow for one Jira ticket inside a target repo. Nothing here is
project-specific; a target opts in with a committed .qa/config.json (a filled-in example is
config/examples/full-example.json).
Docs. Only README.md and this file sit at the root; every other doc is in docs/ (the map
is in README "Docs"). Keep the set small: a new doc needs a purpose no existing one covers.
docs/design.md: how it works and why. Read only the section you need: §4 agents, §5 flow, §6 guard rules, §8 run folder.docs/setup.md: verified facts (versions, hook payloads, Jira, memory), each with its date.docs/dry-run.md: checking a target's setup without Jira or GitHub writes.docs/demo/: an intro deck and the full workflow map (workflow.html).
Every doc states only what is true now, checked against the code, config or git. A past observation is written with its date ("verified 2026-09-24: …"). There is no changelog; git history is the record.
Editing this repo doesn't change the plugin you're running. Where the running copy comes from depends on how it was installed:
- Local-directory marketplace (developing the plugin): the plugin loads in place from the directory in
~/.claude/plugins/known_marketplaces.json(aneskurtovic→installLocation), usually the main checkout, not a worktree. A change goes live only after it's merged tomainand that checkout pulls. The next new session picks it up, with no reinstall. (The install also leaves a copy in~/.claude/plugins/cache/; that copy isn't what loads, seedocs/setup.md.) - GitHub marketplace: the plugin is copied into
~/.claude/plugins/cache/, and a new copy arrives only whenversionchanges (claude plugin marketplace update aneskurtovic).
The plugin is installed per project (--scope project, or --scope local to keep it out of
git), never at user scope. See README "Install".
Commands
npm test # all tests (node --test, no deps)
node --test hooks/test/guard.test.mjs # one file
node --test --test-name-pattern="restore-spec" "scripts/test/*.test.mjs" # one test by name
claude plugin validate . # plugin manifest/agents/skills/hooks
node scripts/check-config.mjs --project <target repo> # check a target's .qa/config.json
QA_HOOK_LOG=1 claude … && node hooks/test/summarize-log.mjs # record payloads, summarize after the session ends- There is no build or lint step and no npm dependencies. Keep it that way: scripts and hooks use only Node built-ins.
- Node ≥ 22 (
enginesinpackage.json). CI runs the suite on Node 22 and 24 onwindows-latest. If you change the floor, change the CI matrix too. (npm testrelies onnode --testexpanding quoted globs, which Node 20 can't do.) - The tests create real git repos in temp dirs, so
git config user.email/user.namemust be set. - CI runs on Windows on purpose: the plugin runs on Windows/Git Bash, and many tests cover path handling.
Architecture
Three layers, each with a different trust level:
- Orchestrator playbook:
skills/qa-run/SKILL.md, executed by the main session as/qa-orchestrator:qa-run <KEY>. It's the only layer that talks to Jira (Atlassian MCP) and asks the human at gates (G0–G3, DELIVER). It setsdisable-model-invocation: true, which means Claude can't start the skill on its own. Only the typed/qa-orchestrator:qa-runcommand can. That matters because the skill moves Jira tickets.skills/setup/SKILL.md(/qa-orchestrator:setup) writes a project's.qa/config.json. It only reads Jira and moves no ticket, so Claude may start it on its own when asked to set the plugin up. It commits, pushes and opens a PR only when the file can't land in the user's checkout (a background job or a worktree) and the user picks that option. - Seven leaf agents in
agents/. Each has a fixed model, which the dispatch must not override (the guard denies it). Each is dispatched with a prompt pointing at a brief (RUN/briefs/<step>.md) thatscripts/brief.mjsfills fromtemplates/briefs/<agent>.md({{var}}placeholders). An agent returns one fenced JSON block, whichscripts/validate.mjschecks againstschemas/<agent>.schema.jsonplus semantic checks: AC ids exist and files are underqa.testDir. - Deterministic scripts in
scripts/do every write that matters: git worktree/branch, state, plan merge, deletes, test runs, delivery (commit/push/PR), metrics, memory.
The guard (hooks/guard.mjs, a PreToolUse hook) enforces rules H1–H11 against the model:
- It applies only to a QA run, through two layers, and both are needed. (1) Install scope: the
plugin is installed at project/local scope, so its hooks load only in the target repo. (2) Run
ownership: inside that repo, every session still loads the hook, including normal dev work.
findOwnedRun()(hooks/lib/context.mjs) therefore must stay fail-open. It allows the call unless the session provably owns an active run (.qa-runs/active.jsonhas thissession_id), and it also allows on any error while checking. After ownership is proven, errors fail closed (exit 2). That includes a missing.qa/config.jsonwhile the run is open, so moving it away can't switch the guard off (a released run goes dormant instead). decide()inhooks/lib/policy.mjsis a pure function of the hook input and the run context. Shell rules live inhooks/lib/shell.mjs: deny rules scan the whole command text, and subagent allowlists must match the whole command.hooks/record.mjs(SubagentStop) appends agent telemetry toRUN/events.jsonland saves the agent's handback toRUN/outputs/<step>.mdforvalidate.mjs --step. It never blocks.- Tests drive the real hook by spawning
guard.mjswith a JSON payload on stdin (callGuard/expectDecisioninhooks/test/guard.test.mjs).
Script conventions (scripts/lib/common.mjs)
- The entry point is
main(fn). It prints the result as JSON and exits 1 on failure, 2 onUsageError. A script signals that a breaker or check fired by exiting 3 itself (preflight.mjs,test-diff.mjs,check-config.mjs,validate.mjs --stepwith no saved output); the JSON says which. The SKILL playbooks branch on these codes. loadConfigreturns.qa/config.jsonwith the defaults fromscripts/lib/config.mjsfilled in; the guard'sloadContextapplies the samewithDefaults.parseArgsturns--kebab-caseflags into camelCase keys.isMain(import.meta.url)guards the CLI, so tests import the exported functions directly.- External commands go through an injected
runner(realRunnerin production, fakes in tests). Never callspawnSyncdirectly in a script. - Writes use
writeJsonAtomic/writeTextAtomic(temp file + rename). scripts/lib/schema.mjsis a subset JSON Schema validator: type, enum, required, properties,additionalProperties: false, items, minItems, minLength, pattern, minimum, maximum. Any other keyword inschemas/is silently ignored.
Run state (lives in the target repo, gitignored: .qa-runs/, .qa-worktrees/)
- The playbook changes
RUN/state.jsononly throughstate.mjs, and everystate.mjswrite callsrefreshViews(scripts/lib/views.mjs), which regeneratesprogress.md,dashboard.htmlandmetrics.json.preflight.mjsanddeliver-tests.mjsalso write their own fields there, without refreshing the views. H10 protects these files, plusplan.json,suite-baseline.json,counters.log,events.jsonl,active.json,.lock,history.jsonl, memory and theoutputs/folder (the lists are inhooks/lib/context.mjs). - G2 approval binds to the
planHashofplan.jsonon disk at that moment. Any later plan edit invalidates it. counters.logis append-only (one JSON line per event), so parallel hook invocations can't lose counts. The H5 (plan revisions) and H7 (test launches) caps are computed from it.
Paths are Windows-first
The same file shows up as C:\a, C:/a and /c/a (Git Bash). Every path comparison in the guard
goes through norm()/isUnder() in hooks/lib/paths.mjs, which also lowercases drive-letter
paths. Playwright file filters must be forward-slash: backslash filters select 0 tests.
Changes that span files
- A guard rule (H1–H11):
hooks/lib/policy.mjs/shell.mjs, a test inhooks/test/guard.test.mjs, the table indocs/design.md§6, the summary in README.md, and any mention inskills/qa-run/SKILL.md(the playbook tells the orchestrator how to react to specific denials). - An agent's output contract:
agents/<name>.md(instructions),schemas/<name>.schema.json,semanticErrorsinscripts/validate.mjs, the consumers of that JSON (merge-plan.mjs,views.mjsdigests,memory.mjs, the next agent's brief), and test fixtures. - A new agent: everything above, plus
templates/briefs/<name>.md, brief variables inbrief.mjs, its shell allowlist (H6) and read scope, and the model list in SKILL.md. - A config key:
scripts/lib/config.mjs(KNOWN, plus its default inwithDefaultsor its entry in therequiredlist),config/examples/full-example.json(andconfig/qa.config.example.jsonif it's required), the tables inskills/setup/SKILL.mdsteps 2–3, and the config table indocs/design.md§3. A new default belongs inwithDefaults, not at the use site: that's how the scripts and the guard see the same value. - A playbook step or gate:
skills/qa-run/SKILL.md, the flow and gate table indocs/design.md§5, and the step comment at the top of the script it calls. - A release: bump the version in both
package.jsonand.claude-plugin/plugin.json. Adopters on the GitHub marketplace get the change only after a version bump. - Node floor:
enginesinpackage.json, the CI matrix in.github/workflows/test.yml, and the README requirements line. - Install scope or instructions: README "Install",
docs/setup.md,docs/design.md§3 / §6, thehooks/hooks.jsondescription, thehooks/guard.mjsheader comment, and this file. - Moving or renumbering a doc section: grep for its path and
§number. The script and hook header comments citedocs/design.mdsections.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

