sumo-qa
Copilot instructions7 starsChanged 40 days agoApache-2.0View on GitHub
Hard rule: for any QA-shaped request — testing, test plan, test strategy, test approach, regression scope, risk-based testing, exploratory testing, code review for safety-to-merge, scaffolding tests, TDD, mutation testing, finding or validating test data, QA audit, test pyramid design — you MUST call using_sumo_qa (the entry router) before answering. Do not produce QA advice from general training-data knowledge. When citing principles, techniques, classifications, or approaches, load them first via the sumo_qa_load_* tools and cite the loaded catalogue verbatim; say "not…
Where files are headingNew Copilot instructions files have more rules about branches and pull requests, and fewer about code style.
Of the Copilot instructions files we read, the 2,349 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.
In this repository21 other files it gives its agents.
- AGENTS.mdAGENTS.md
- regen-eval-baselineSkill
- scaffold-sumo-qa-skillSkill
- sumo-qa-answering-testing-questionSkill
- sumo-qa-closing-qa-gapsSkill
- sumo-qa-creating-test-planSkill
- sumo-qa-deciding-approachSkill
- sumo-qa-executing-qa-rolloutSkill
- sumo-qa-finding-test-dataSkill
- sumo-qa-finishing-qa-workSkill
- sumo-qa-implementing-with-tddSkill
- sumo-qa-measuring-coverageSkill
- sumo-qa-planning-qa-rolloutSkill
- sumo-qa-preparing-for-workSkill
- sumo-qa-reviewing-before-mergeSkill
- sumo-qa-security-testingSkill
- sumo-qa-strategisingSkill
- sumo-qa-strengthening-testsSkill
- sumo-qa-suggesting-external-skillSkill
- sumo-qa-triaging-test-failuresSkill
- using-sumo-qaSkill
The file
We split this copilot-instructions.md into the rules it gives an agent
1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 62
When creating GitHub issues, use the closest template in .github/ISSUE_TEMPLATE/ and include every required field in the issue body.
≈ “Common” is an estimate: at least 1 in 5 of the 3,779 Copilot instructions files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
QA tasks via sumo-qa MCP
Hard rule: for any QA-shaped request — testing, test plan, test strategy,
test approach, regression scope, risk-based testing, exploratory testing,
code review for safety-to-merge, scaffolding tests, TDD, mutation testing,
finding or validating test data, QA audit, test pyramid design — you MUST
call using_sumo_qa (the entry router) before answering. Do not
produce QA advice from general training-data knowledge. When citing
principles, techniques, classifications, or approaches, load them first
via the sumo_qa_load_* tools and cite the loaded catalogue verbatim;
say "not in the catalogue" rather than supplementing from memory.
First hop: every QA-shaped request, including a development-framed one such
as "I'm adding X, how should I test it?", "what tests do I need?" or "write
the failing tests first", calls using_sumo_qa before any other sumo-qa tool
and before any QA advice, then sumo_qa_deciding_approach, then the one skill
it routes to. No specialist skill is entered directly.
An underspecified QA request still takes the first hop before you ask the user
anything; sumo_qa_deciding_approach, or the skill it routes to, asks the one
clarifying question it needs.
For QA-shaped requests in this repo (test plans, code review, scaffolding
tests, finding test data, deciding QA approach), call the relevant skill tool
on the sumo-qa MCP server and follow the checklist it returns.
Available skills (each registered as an MCP tool named after the skill,
hyphens replaced by underscores). Every skill after the first two is reached
through sumo_qa_deciding_approach, never called as the first hop:
using_sumo_qa— entry router; load this first for any QA intentsumo_qa_deciding_approach— pick the QA approach for the worksumo_qa_preparing_for_work— plan QA before coding startssumo_qa_creating_test_plan— produce entry/exit criteria, phases, deliverablessumo_qa_implementing_with_tdd— red-green-refactor cyclesumo_qa_reviewing_before_merge— review local diffsumo_qa_strengthening_tests— mutation-testing follow-upsumo_qa_finding_test_data— known-good test data discovery and validationsumo_qa_answering_testing_question— generic "how do I test this?"sumo_qa_strategising— repo-wide QA strategysumo_qa_planning_qa_rollout— turn a QA chunk into a written plan of bite-sized taskssumo_qa_executing_qa_rollout— dispatch a signed-off plan task-by-task to fresh subagentssumo_qa_finishing_qa_work— capture evidence and write a PR-ready summarysumo_qa_measuring_coverage— run the repo's coverage/mutation tools, collect the results, persist them for the QA reportsumo_qa_suggesting_external_skill— fallback when no native skill fits the intentsumo_qa_closing_qa_gaps— close a named uncovered-behavior gap, one evidenced loop at a timesumo_qa_triaging_test_failures— triage a failing/flaky test: classify the cause, name the smallest isolation step before any fixsumo_qa_security_testing: select deeper security evidence for explicit security testing or a grounded security gap
The skills carry the senior-QA discipline (Iron Laws, checklists, Red Flags).
Knowledge catalogues are accessed via the sumo_qa_load_* tools — use them
for principles, techniques, classifications, approaches, and specialty tool
fits before relying on training-data knowledge.
Repo-map accelerators: when a .sumo-qa/repo-map.json artifact is present,
sumo_qa_scan_repo, sumo_qa_analyze_diff_impact, and sumo_qa_query_repo_map
provide fast, deterministic evidence over the codebase (inventory, changed-file
impact, ranked search) for the review, strategy, and prepare-for-work skills.
sumo_qa_generate_qa_report composes the persisted .sumo-qa artifacts into
the static local QA report (.sumo-qa/qa-report.html) when the user asks for
a QA report or dashboard.
When creating GitHub issues, use the closest template in .github/ISSUE_TEMPLATE/
and include every required field in the issue body. For implementation tasks
raised by an AI agent, use .github/ISSUE_TEMPLATE/ai-task.yml unless a more
specific template fits better.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

