test-strategy
Skill0 starsChanged 3 months agoApache-2.0View on GitHub
Decide what to test at which layer, and audit whether existing tests actually constrain behavior. Use when planning tests for a story, running the G4 QA gate, reviewing test quality in a PR, deciding between unit and integration coverage, or when asked whether coverage is meaningful. Includes the delete-the-implementation audit and flaky-test handling.
41 other projects use a version of this.
Adopted by
Most recently taken up by
- meaningfy-ws/skillerytheir own version, first seen Nov 16, 2026
- swap-cameroon/kuro-antigravity-configtheir own version, first seen Nov 15, 2026
- l-gevity/l-gevity-skillstheir own version, first seen Nov 14, 2026
- frnkgns/Overgeared-Skillstheir own version, first seen Nov 8, 2026
- reddygopalkrishna/developer-agent-skillstheir own version, first seen Nov 8, 2026
- hatemecha/skillstheir own version, first seen Nov 1, 2026
Used with
Projects with it also use database-testing, data-pipelines and 8 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
petrkindlmannSkilldatabase-testing
Validate database integrity, test migrations forward and backward, verify schema constraints, manage seed data, detect migration drift, and identify query performance issues. Covers PostgreSQL, MySQL, MongoDB with Prisma, TypeORM, Drizzle, and SQLAlchemy, plus Testcontainers test databases. Use when: "database test," "migration test," "migration rollback," "rollback test," "data integrity," "SQL test," "schema validation," "seed data," "query performance," "Testcontainers." Not for: synthetic data generation/masking at scale — use test-data-management; Docker/IaC test-environment provisioning — use test-environments; SQL injection — use security-testing. Related: test-data-management, test-environments, security-testing, ci-cd-integration.
Also in 3 projects that use this
YepAPISkilldata-pipelines
ETL patterns, cron scheduling, queue-based processing, and batch jobs with Inngest/BullMQ.
Also in 3 projects that use this
ffsshhttiikkSkillunit-testing
Unit testing best practices and patterns
Also in 4 projects that use this
AnthropicSkillincident-response
Run an incident response workflow — triage, communicate, and write postmortem. Trigger with "we have an incident", "production is down", an alert that needs severity assessment, a status update mid-incident, or when writing a blameless postmortem after resolution.
Also in 7 projects that use this
OpenAISkillobservability
Vercel Observability expert guidance — Drains (logs, traces, speed insights, web analytics), Web Analytics, Speed Insights, runtime logs, custom events, OpenTelemetry integration, and monitoring dashboards. Use when instrumenting, debugging, or optimizing application performance and user experience on Vercel.
Also in 7 projects that use this
Microsoft AzureSkillrelease-readiness
Complete ODIN release documentation and validation checks before declaring user-visible work complete, preparing a release, or opening the Release-to-main pull request.
Also in 4 projects that use this
Other versions41 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 41
- nurettincoban/ai-prd-workflow★ 300same name
- petrkindlmann/qa-skills★ 274same name
- kid-sid/claude-spellbook★ 190same name
- avelikiy/great_cto★ 115same name
- l-gevity/l-gevity-skills★ 88same name
- fishzjp/qa-skills★ 64same name
- nahid-sparktales/agent-dispatcher★ 59same name
- nimadorostkar/Claude-Skills-collection★ 37same name
- caiaffa/claude-code-ultimate-engineering-system★ 17same name
- jpantsjoha/ai-native-developer-experience★ 13same name
31 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository18 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- ai-impact-assessmentSkill
- artifact-intakeSkill
- backlog-prioritizationSkill
- blameless-postmortemSkill
- control-assessmentSkill
- control-verificationSkill
- dependency-vettingSkill
- eval-designSkill
- evidence-writingSkill
- exploratory-testingSkill
- red-teamingSkill
- retro-facilitationSkill
- scanner-triageSkill
- secure-code-reviewSkill
- story-splittingSkill
- threat-modelingSkill
- writing-acceptance-criteriaSkill
The file
We split this skill into the rules it gives an agent
3 rules. 0 are common, 3 are distinctive, and 0 are very specific to this project.
0
Common
Most of the 36 edited versions we read keep these.
3
Distinctive
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 52
Only the second is SA-11 security evidence.
- Distinctive: line 61
If a plausible bug would leave a test green, it is decorative — and worse than absent, because it inflates the coverage number and buys …
- Distinctive: line 95
Never retry it away. The retry hides a real race condition, and the race is the finding — it is telling you something true about the system …
“Common” is measured: kept by at least 6 in 10 of the 36 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Test strategy
Consumers: qa-engineer, /qa-gate, developer, /review, process/gates/g4-verified.md.
Shape
╱╲ Exploratory — charter-driven, time-boxed. Finds what nobody imagined.
╱ ╲
╱────╲
╱ E2E ╲ Few. Critical journeys only. Slow, brittle — and irreplaceable for
╱────────╲ proving the whole thing hangs together.
╱Integration╲ Real collaborators: DB, queue, HTTP boundary, authz. This layer catches
╱────────────╲ what unit tests mock away, which is most real defects.
╱ Unit ╲ Many, fast, deterministic. Pure logic, branches, boundaries.
╱────────────────╲Integration is the layer most teams under-invest in. Unit-only suites produce systems where every part passes and the whole fails. E2E-heavy suites produce slow, flaky suites nobody trusts and everyone re-runs until green — which is worse than no suite, because it still shows green.
Placement
| Concern | Layer | Why |
| Business logic, branches, boundaries | Unit | Fast, exhaustive on the interesting cases |
| Authorization decisions | Integration — always | Unit-mocked authz proves the mock, not the authorizer |
| Persistence, transactions, migrations | Integration, real engine | An in-memory substitute has different semantics exactly where it matters |
| Contract with an external service | Contract test + recorded-fixture integration | |
| Critical user journey | E2E — one per journey, no more | |
| Input validation | Property-based where the space is large | |
| Performance NFR | Load test, threshold as the assertion, representative data volume | Performance on an empty database is not verified |
| Accessibility | Automated axe scan + manual keyboard and screen-reader pass | The scan catches maybe half |
| A security control from the threat model | Integration, negative case | See below |
The negative-case rule
For every control, the test that matters is the one proving it denies.
Positive: owner reads their draft → 200 ← the feature works
Negative: non-owner reads it → 403 ← the CONTROL worksOnly the second is SA-11 security evidence. A story whose controls have only positive tests fails G4 regardless of coverage percentage.
The test-quality audit
Judge tests by whether they constrain behavior. The check:
Mentally delete the implementation. Which tests fail?
If a plausible bug would leave a test green, it is decorative — and worse than absent, because it inflates the coverage number and buys false confidence.
Watch for:
- Asserting implementation detail rather than observable behavior — passes with the logic removed, fails on harmless refactors. Exactly backwards.
- Assertions too loose to fail —
assert result is not None,assert len(x) > 0. - Setup so elaborate the test's meaning is unrecoverable; nobody will maintain it and everybody will skip it.
- Shared mutable state between tests — passes in isolation, fails in CI, or vice versa.
- Wall-clock or network dependence.
- Tests that mirror the implementation's structure — they were written by reading the code, so they encode the same misunderstanding.
Report the audit as a finding, specifically:
"AC-5's test asserts internal eviction ordering rather than observable behavior. It passes with the eviction logic removed entirely. Not valid coverage."
Coverage
A floor, not a target. Coverage measures reach, not quality — 100% coverage of assertions that cannot fail is 0% verification.
Two rules worth enforcing in CI:
- Absolute threshold met
- No drop versus the base branch — otherwise a large untested change passes by diluting an already-high number
Flaky tests
A flaky test is a defect. File it as one.
Never retry it away. The retry hides a real race condition, and the race is the finding — it is telling you something true about the system that you are choosing not to hear. A suite with retries configured is a suite that has stopped being evidence.
Traceability
Build AC → test → result. An AC with no test fails G4. A test with no AC is either dead weight or an undocumented requirement — resolve which; do not leave it ambiguous.
| AC | Test | Type | Result | Evidence |
|----|------|------|--------|----------|
| AC-1 draft offered on re-auth | test_draft_offered_after_expiry | Integration | ✅ | run #4821 |
| NFR authz | test_other_user_cannot_read_draft | Integration (negative) | ✅ | run #4821 |
| NFR p95 < 200 ms | load_draft_restore | Performance | ⚠️ 240 ms | run #4823 |Non-functional verification
- Performance — the NFR number is the assertion, under representative load and data volume
- Reliability — exercise dependency failure deliberately. Kill the DB mid-transaction and watch. Make the dependency slow rather than down; that path is usually untested and usually worse.
- Recovery — roll back for real, then roll forward. An untested rollback is a hypothesis.
- Observability — assert the required log, metric, or trace actually appears
Reporting
Report results exactly as observed. If unverified, say unverified — never "passed" for something you did not run. A QA sign-off that overstates coverage is the most damaging artifact in this process, because everything downstream trusts it.
Reject specifically, naming the criterion and the observation. "Needs more testing" is not a rejection.
Controls
SA-11 · SA-11(1) · SA-11(8) · CA-2 · CM-4 · AU-2 (audit event verification).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

