qa-orchestrator
Skill1 starsChanged yesterdayMITView on GitHub
Take a change through the QA gates - Ready, Designed, Built and tested, Verified in CI, Release ready, Deployed, Hypercare (configurable) - by running the right skill at each step, recording each report with its SHA-256 and its own validator's result, evaluating the gate's exit criteria, and asking the named person to approve, approve with an override, or reject before moving on. Run state lives in a temporary folder; a script keeps the state and refuses an approval without evaluated criteria, a person, or (when a criterion failed) an override reason. Never auto-approves unless the configuration says so, never overrides a step skill's own rules, and never calls QA complete.
3 other projects use a version of this.
Adopted by
Most recently taken up by
- jianjian12138/test-skillstheir own version, first seen Oct 25, 2026
- romangalaxys10-spec/fabletheir own version, first seen Oct 13, 2026
- AZANIR/qa-skillstheir own version, first seen Mar 23, 2026
Other versions3 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 3
- romangalaxys10-spec/fable★ 4same name
- AZANIR/qa-skills★ 2same name
- jianjian12138/test-skills★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository33 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- api-contract-testerSkill
- bug-reproducerSkill
- ci-failure-triagerSkill
- ci-suite-runnerSkill
- code-security-auditorSkill
- cross-browser-checkerSkill
- db-migration-reviewerSkill
- dead-test-finderSkill
- exploratory-charter-writerSkill
- flaky-test-hunterSkill
- hypercare-anomaly-watcherSkill
- i18n-checkerSkill
- journey-mapperSkill
- journey-to-testsSkill
- mobile-app-auditorSkill
- mutation-testerSkill
- performance-budget-auditorSkill
- post-deploy-validatorSkill
- privacy-flow-auditorSkill
- release-readiness-gateSkill
- requirements-traceability-auditorSkill
- resilience-proberSkill
- risk-based-test-plannerSkill
- test-case-designerSkill
- test-data-auditorSkill
- test-impact-selectorSkill
- test-suite-evaluatorSkill
- ticket-readiness-reviewerSkill
- ticket-to-tddSkill
- ui-crash-hunterSkill
- visual-regression-reviewerSkill
- wcag-auditorSkill
The file
We split this skill into the rules it gives an agent
4 rules. About 0 are common, 2 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 34
Step skills keep their own rules: Run each step by its skill's SKILL.md: the CI suite runner still needs its trigger confirmation, the …
- Distinctive: line 62
On rejection: stop; say what would reopen the gate (a fresh report, a fix).
- Distinctive: line 63
Summarise: when asked or at the end.
- Very specific: line 84
The person approves a gate with failed criteria: ask for the reason and pass it as --override.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
QA orchestrator
Mission
Sequence the QA skills across a change's gates and keep a person in the loop:
| Gate | Steps (default) | Exit criteria (default) |
| G1 Ready | ticket-readiness-reviewer | review valid; verdict ready or ready with questions |
| G2 Designed | test-case-designer (risk-based-test-planner optional) | cases valid; every design row covered |
| G3 Built and tested | ticket-to-tdd, code-security-auditor | reports valid; no P0 security finding |
| G4 Verified in CI | ci-suite-runner (ci-failure-triager optional) | run valid; no failed tests |
| G5 Release ready | release-readiness-gate | gate report valid; verdict go |
| G6 Deployed | post-deploy-validator | report valid; recommendation go |
| G7 Hypercare | hypercare-anomaly-watcher | watch valid; no incident drafts |
The gates, steps, criteria and approvers are in data/orchestrator.json and can be changed per team.
Non-negotiable rules
- A person approves each gate (who and how), unless the config turns on
auto_approvefor that gate; an auto-approval is recorded as such. Off by default everywhere. - Criteria before approval. Evaluate first. A failed criterion needs an override reason to approve; a rejection needs a reason and stops the run until someone reopens the gate.
- In order. A gate starts only after the previous one is approved or skipped (with a reason).
- Reports are evidence: a step counts only with its report recorded (SHA-256) and its validator exiting 0; a report changed afterwards must be recorded again.
- Step skills keep their own rules. Run each step by its skill's
SKILL.md: the CI suite runner still needs its trigger confirmation, the post-deploy validator stays read-only, the release gate never calls a release safe. A gate approval is not any of those confirmations. - State in a temporary folder (
<temp>/qa-orchestrator/<run id>/), not the repository; export a summary only when asked. - No "QA complete" or "approved by QA": the summary lists each gate's decision, who made it, and on what evidence.
Workflow
python scripts/orchestrate.py init --subject "SHOP-310 for release 2026.12" [--config orchestrator.json]
python scripts/orchestrate.py next --state <dir>
# run the step's skill (its SKILL.md), then:
python scripts/orchestrate.py record --state <dir> --gate G1 --step readiness --report review.json
python scripts/orchestrate.py evaluate --state <dir> --gate G1
# ask the gate's approver; record their words:
python scripts/orchestrate.py approve --state <dir> --gate G1 --by "QA lead" --how "in chat" [--override "why"]
python scripts/orchestrate.py reject --state <dir> --gate G5 --by "Release manager" --how "release meeting" --reason "..."
python scripts/validate_run.py <dir>
python scripts/render_report.py <dir> [--markdown summary.md]- Init: the subject (ticket, release, repository, commit) and the config; start later with
--start-gateand--skip-reasonwhen earlier gates were done elsewhere. - For each gate:
next; run each step's skill;recordits report;evaluate; show the person the criteria results and the reports' headlines; record their decision exactly as given. Never decide for them, and never treat silence or a general "carry on" as an approval of a gate with failed criteria. - On rejection: stop; say what would reopen the gate (a fresh report, a fix).
- Summarise when asked or at the end.
Rating
See RUBRIC.md: gate statuses, criteria kinds, decisions.
Output
<state>/state.json (schemas/run_state.schema.json): subject, config and its hash, gates (status, steps
with report, hash and validator result, criteria results, decision, history) and the event log.
Edge cases
- A step's report comes from CI or another person: record it the same way; its validator still runs.
- A validator needs its skill's folder: set the step's
cwdin the config. - Validator commands are never taken from a state file.
recordandvalidate_run.py --rerunrebuild each command from the config file (it must still have the SHA-256 recorded atinit) and run it only when it ispython <skill>/scripts/validate_*.py ...inside the trusted skills folder (the folder holding this skill, or--skills-root), with this Python.--rerunalso needs every recorded command to equal the rebuilt one and every report unchanged; otherwise it refuses (exit 2) before running anything. A state folder someone else wrote is data: validate it, but never edit its commands to make a rerun pass. - The person approves a gate with failed criteria: ask for the reason and pass it as
--override. - Reopening:
reopenwith who and why; the earlier decision stays in the gate's history.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

