test-strategy
Skill88 starsChanged 3 days agoMITView on GitHub
Designs and audits risk-driven test strategies that map grounded requirements, acceptance criteria, quality risks, architecture, and escaped defects to the smallest sufficient evidence portfolio. Use when deciding what to test, which design technique or test fidelity to use, when to use real dependencies versus fakes, stubs, or mocks, how to manage test data and environments, or whether unit, component, integration, contract, E2E, exploratory, property, fuzz, performance, resilience, or recovery testing is justified. Also use to audit hollow assertions, redundant coverage, flaky tests, and residual risk. Do not use for framework-specific test implementation, pipeline placement or gating, or trace evidence state; hand those to stack skills, defect-shift-left, ci-cd-reliability-architecture, and requirements-traceability.
41 other projects use a version of this.
Adopted by
Most recently taken up by
- meaningfy-ws/skillerytheir own version, first seen Nov 16, 2026
- swap-cameroon/kuro-antigravity-configtheir own version, first seen Nov 15, 2026
- frnkgns/Overgeared-Skillstheir own version, first seen Nov 8, 2026
- reddygopalkrishna/developer-agent-skillstheir own version, first seen Nov 8, 2026
- hatemecha/skillstheir own version, first seen Nov 1, 2026
- lelborn/skillstheir own version, first seen Nov 1, 2026
Used with
Projects with it also use database-testing, exploratory-testing and 10 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
petrkindlmannSkilldatabase-testing
Validate database integrity, test migrations forward and backward, verify schema constraints, manage seed data, detect migration drift, and identify query performance issues. Covers PostgreSQL, MySQL, MongoDB with Prisma, TypeORM, Drizzle, and SQLAlchemy, plus Testcontainers test databases. Use when: "database test," "migration test," "migration rollback," "rollback test," "data integrity," "SQL test," "schema validation," "seed data," "query performance," "Testcontainers." Not for: synthetic data generation/masking at scale — use test-data-management; Docker/IaC test-environment provisioning — use test-environments; SQL injection — use security-testing. Related: test-data-management, test-environments, security-testing, ci-cd-integration.
Also in 3 projects that use this
testified-ossSkillexploratory-testing
Guide exploratory testing sessions including charter creation, session-based testing, heuristics application, and finding documentation. Use when planning exploratory testing, conducting test sessions, or documenting discoveries.
Also in 4 projects that use this
YepAPISkilldata-pipelines
ETL patterns, cron scheduling, queue-based processing, and batch jobs with Inngest/BullMQ.
Also in 3 projects that use this
ffsshhttiikkSkillunit-testing
Unit testing best practices and patterns
Also in 4 projects that use this
AnthropicSkillincident-response
Run an incident response workflow — triage, communicate, and write postmortem. Trigger with "we have an incident", "production is down", an alert that needs severity assessment, a status update mid-incident, or when writing a blameless postmortem after resolution.
Also in 7 projects that use this
OpenAISkillobservability
Vercel Observability expert guidance — Drains (logs, traces, speed insights, web analytics), Web Analytics, Speed Insights, runtime logs, custom events, OpenTelemetry integration, and monitoring dashboards. Use when instrumenting, debugging, or optimizing application performance and user experience on Vercel.
Also in 7 projects that use this
Other versions41 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 41
- nurettincoban/ai-prd-workflow★ 300same name
- petrkindlmann/qa-skills★ 274same name
- kid-sid/claude-spellbook★ 190same name
- avelikiy/great_cto★ 115same name
- fishzjp/qa-skills★ 64same name
- nahid-sparktales/agent-dispatcher★ 59same name
- nimadorostkar/Claude-Skills-collection★ 37same name
- caiaffa/claude-code-ultimate-engineering-system★ 17same name
- jpantsjoha/ai-native-developer-experience★ 13same name
- Mozurok/fhorja.dev★ 10same name
31 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository52 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- alchemy-orchestrateSkill
- alchemySkill
- architecture-as-code-javascriptSkill
- architecture-as-code-pythonSkill
- architecture-as-codeSkill
- architecture-guidelinesSkill
- bring-downSkill
- ci-cd-reliability-architectureSkill
- continuous-improvementSkill
- defect-shift-leftSkill
- dependency-lifecycleSkill
- evolutionary-database-designSkill
- functionality-complexity-tradeoffSkill
- gate-lifecycleSkill
- implementation-readinessSkill
- infrastructure-fitSkill
- morphogenetic-architectureSkill
- observability-designSkill
- push-outSkill
- requirements-groundingSkill
- requirements-topologySkill
- requirements-traceabilitySkill
- structural-simplificationSkill
- system-optimizationSkill
- test-strategySkill
- zero-copy-requirementsSkill
- alchemy-orchestrateSkill
- alchemySkill
- architecture-as-code-javascriptSkill
- architecture-as-code-pythonSkill
- architecture-as-codeSkill
- architecture-guidelinesSkill
- bring-downSkill
- ci-cd-reliability-architectureSkill
- continuous-improvementSkill
- defect-shift-leftSkill
- dependency-lifecycleSkill
- evolutionary-database-designSkill
- functionality-complexity-tradeoffSkill
- gate-lifecycleSkill
- implementation-readinessSkill
- infrastructure-fitSkill
- morphogenetic-architectureSkill
- observability-designSkill
- push-outSkill
- requirements-groundingSkill
- requirements-topologySkill
- requirements-traceabilitySkill
- structural-simplificationSkill
- system-optimizationSkill
- zero-copy-requirementsSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
29 rules. 0 are common, 26 are distinctive, and 3 are very specific to this project.
0
Common
Most of the 36 edited versions we read keep these.
26
Distinctive
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 26
Risk before test type: Start from a requirement, failure mode, quality risk, incident, or contract.
- Distinctive: line 30
Minimum sufficient fidelity: Use the smallest scope that can faithfully observe the named failure.
- Distinctive: line 44
Use this skill after a requirement or remediation increment has usable completion conditions, or retrospectively when an existing behavior, …
- Distinctive: line 57
Use a Combined pass only for Audit mode against a stable architecture, or when the relevant architecture is already accepted and will not …
- Very specific: line 73
If requirement meaning, actors, permissions, or completion conditions are unclear, return to requirements-grounding or …
- Distinctive: line 93
Use this Alchemy hand-off when both passes apply:
- Distinctive: line 107
Do not restart the Obligation pass after architecture unless the architecture changes requirement meaning, a failure mode, or an oracle.
- Distinctive: line 144
Keep impact, exposure, and evidence confidence separate.
- Very specific: line 159
If no credible oracle exists, emit DEFER for that obligation and name the missing requirement, contract, signal, or human judgment.
- Distinctive: line 200
Only then apply labels such as unit, component, integration, contract, system, E2E, or production probe.
- Distinctive: line 208
Prefer the smallest candidate whose fidelity can observe the failure.
- Distinctive: line 214
Read the same reference for test doubles, production-compatible dependencies, test data, environment isolation, quarantine policy, adequacy …
- Distinctive: line 220
Use real, deterministic, in-process owned collaborators.
- Distinctive: line 223
Use a mock only when the interaction itself is required behavior.
- Distinctive: line 233
Check the applicable surface without inventing requirements:
- Distinctive: line 246
When a quality area is material but lacks an authoritative requirement or specialist policy, record a blocker or route to the matching …
- Distinctive: line 248
Do not manufacture thresholds.
- Distinctive: line 258
Treat every escaped defect as evidence that the portfolio missed a risk, technique, oracle, fidelity, or placement decision.
- Distinctive: line 260
Treat flakes, order dependence, uncontrolled time, shared state, and environment leakage as test defects.
- Distinctive: line 262
Use coverage as a gap locator.
- Distinctive: line 264
Remove a redundant broader test only after the smaller replacement proves the same scope.
- Very specific: line 297
Do not emit ADEQUATE from test counts, pyramid shape, coverage percentage, green CI, or one successful release.
- Distinctive: line 327
Do not invent requirement meaning, thresholds, users, traffic, or risk acceptance.
- Distinctive: line 329
Do not use a test framework preference as a strategy.
- Distinctive: line 330
Do not require every technique for every system.
- Distinctive: line 331
Do not equate mocks with isolation or real dependencies with confidence.
- Distinctive: line 332
Do not automate subjective evaluation merely to make it countable.
- Distinctive: line 333
Do not retain expensive tests because they were expensive to create.
- Distinctive: line 334
Do not claim an ownership boundary listed under Boundary.
“Common” is measured: kept by at least 6 in 10 of the 36 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Test Strategy
Convert accepted behavior and quality risks into the smallest reliable evidence portfolio. Choose tests by the failure they must detect, not by a universal pyramid ratio, coverage target, or preferred tool.
Core Directives
- Risk before test type. Start from a requirement, failure mode, quality risk, incident, or contract. Never start from “we need more unit tests.”
- Oracle before harness. State how correct and incorrect behavior can be distinguished before selecting a test level, framework, or environment.
- Minimum sufficient fidelity. Use the smallest scope that can faithfully observe the named failure. Add broader tests only for distinct confidence.
- Evidence over percentages. Coverage locates unexecuted code; it does not prove useful assertions, satisfied requirements, or acceptable residual risk.
- Real boundaries where semantics matter. A fast double cannot prove database, wire, framework, migration, or provider compatibility.
- Reliability is part of validity. A flaky or order-dependent result is not trustworthy verification.
- Keep unknowns visible. Record untested risks, manual obligations, missing environments, and accepted residual risk instead of silently marking them covered.
Boundary
Use this skill after a requirement or remediation increment has usable completion conditions, or retrospectively when an existing behavior, incident, or test portfolio provides a bounded subject. In Design mode, use two passes when architecture can change boundaries, dependency semantics, deployment topology, or observability:
- Obligation pass — after readiness, before A. Define risks, failure modes, oracles, and required confidence without freezing architecture- dependent scope or fidelity.
- Portfolio pass — after final A/L/C and E when applicable, before H. Consume the accepted architecture and finalize technique, scope, fidelity, dependencies, data, environment, and stimulus.
Use a Combined pass only for Audit mode against a stable architecture, or when the relevant architecture is already accepted and will not change.
Consume:
- canonical requirement and acceptance-criterion IDs when available;
- actor workflows, contracts, current or accepted architecture boundaries, and data ownership;
- functional and quality constraints;
- the readiness aspect matrix, at capability level, for the Obligation pass;
- the subsystem-level governed set of each aspect — from the accepted placement decision or the architecture-as-code registry — for the Portfolio pass;
- historical defects, incidents, change hotspots, and production signals;
- delivery, environment, privacy, compliance, and cost constraints.
If requirement meaning, actors, permissions, or completion conditions are
unclear, return to requirements-grounding or implementation-readiness.
Existing tests and code are evidence, not product intent.
This skill owns:
- the risk-to-evidence mapping;
- the oracle, technique, scope, fidelity, dependency, data, and environment decisions for each test obligation;
- portfolio adequacy, duplication, and residual-risk analysis.
This skill does not own:
- requirement meaning or readiness;
- implementation recipes for a specific test framework;
- the earliest pipeline stage, job structure, blocking action, or evidence freshness;
- the distinction between implemented and executed/verified evidence;
- specialist security, accessibility, safety, or domain policy.
Use this Alchemy hand-off when both passes apply:
Implementation Readiness
→ Test Strategy — Obligation pass: risks, failure modes, oracles, confidence
→ A → L/C → E, as justified: accepted architecture and enforcement
→ Test Strategy — Portfolio pass: technique, scope, fidelity, dependencies,
data, environment, stimulus
→ Defect Shift-Left: earliest capable stage
→ CI/CD Reliability: execution trigger, gate, artifact, freshness, failure action
→ Requirements Traceability: implementation and executed-evidence state
→ System Optimization: suite flow, cost, duplication, and bottlenecksDo not restart the Obligation pass after architecture unless the architecture changes requirement meaning, a failure mode, or an oracle. If accepted architecture changes after the Portfolio pass, rerun only the affected portfolio rows before handing them to H.
test-strategy is a task-matched Alchemy companion. It is not a new
A.L.C.H.E.M.Y. letter, qualification stage, or gate.
Workflow
1. Qualify the subject
Select a mode:
- Design — derive evidence obligations for admitted or clearly bounded work.
- Audit — assess an existing portfolio against current requirements, risks, incidents, and architecture.
In Design mode, also select the strategy pass defined under Boundary: Obligation, Portfolio, or Combined.
Define the subject as a requirement increment, subsystem, contract, workflow, change, release risk, or bounded system. Do not strategize for “the whole application” when no risk or decision boundary is stated.
2. Build the risk ledger
For each material risk, record:
Risk / criterion: <canonical ID or named risk>
Failure mode: <observable wrong outcome>
Impact: <user, business, security, operational, or compliance harm>
Exposure: <where and how plausibly it occurs>
Risk evidence: <requirement, incident, telemetry, contract, or assumption>Keep impact, exposure, and evidence confidence separate. Do not collapse them into one score. Unknown exposure is not low exposure. External legal, contractual, accessibility, safety, or security obligations remain required even when usage is low.
3. Define the oracle
For every required obligation, state:
- the source of truth;
- the observable pass and fail condition;
- tolerances, time windows, ordering, or eventual-consistency rules;
- false-pass and false-fail risks;
- the data and instrumentation needed to observe the result.
If no credible oracle exists, emit DEFER for that obligation and name the
missing requirement, contract, signal, or human judgment. A test that merely
runs code or asserts “not null” is not evidence unless that is the actual
criterion.
4. Select the technique
Read references/technique-selection.md when the technique is not obvious, the subject has several failure modes, or property, model-based, contract, fuzz, exploratory, performance, resilience, or recovery testing may apply.
Select the technique that attacks the risk directly:
- examples for named business cases;
- partitions and boundaries for classified input ranges;
- decision tables or combinations for interacting rules;
- state-transition or model-based tests for workflows;
- property, metamorphic, or differential tests for invariants and broad input;
- contract or real-boundary integration tests for compatibility;
- a few E2E tests for irreducible critical journeys;
- exploratory or user evaluation for unknown, subjective, or human-centered risks.
Record why each rejected technique would add less confidence or more cost. In an Obligation pass, stop at the architecture-independent technique family; the Portfolio or Combined pass fixes the concrete harness and scope.
5. Select scope and fidelity
Describe every proposed test by its:
System under test | exercised dependencies | environment | stimulus | oraclestimulus is the input, actor action, event, state transition, elapsed time, or
injected fault exercised by the test. It is not the CI/CD execution trigger
(push, pull request, schedule, manual dispatch, deployment, or production
event), which belongs to ci-cd-reliability-architecture.
Only then apply labels such as unit, component, integration, contract, system, E2E, or production probe. Labels vary across teams; the five dimensions above must not.
Compare candidate scopes on speed, maintainability, utilization, reliability, and fidelity with the scorecard in references/portfolio-governance.md.
Prefer the smallest candidate whose fidelity can observe the failure. Keep a broader test only when it covers composition, environment, or behavior the smaller test cannot.
6. Decide dependencies, data, and environment
Read the same reference for test doubles, production-compatible dependencies, test data, environment isolation, quarantine policy, adequacy signals, and suite budgets.
Apply these defaults:
- Use real, deterministic, in-process owned collaborators.
- Stub slow, nondeterministic, destructive, or unavailable side effects when testing local decisions.
- Use a mock only when the interaction itself is required behavior.
- Exercise the real dependency when compatibility or provider semantics are the risk.
- Protect every double that represents an external boundary with a shared schema, executable contract, or real-boundary backstop.
- Make data self-contained, resettable, privacy-safe, versioned where needed, and provisionable in parallel.
7. Complete the quality scan
Check the applicable surface without inventing requirements:
- functional correctness and actor outcomes;
- compatibility, concurrency, version skew, and migrations;
- performance, capacity, and resource use;
- reliability, resilience, recovery, and data integrity;
- security, privacy, compliance, and auditability;
- usability, accessibility, and supported environments;
- operability, configuration, deployment, and production verification;
- aspects: for every aspect that holds across the subject, one oracle per governed subsystem or one generative check over the governed set — a hand-written test per cell is the n-copies smell.
When a quality area is material but lacks an authoritative requirement or specialist policy, record a blocker or route to the matching companion skill. Do not manufacture thresholds.
8. Audit the portfolio
In Audit mode:
- Map current tests to risks, criteria, boundaries, and observed incidents.
- Find missing obligations, hollow oracles, stale tests, unverified doubles, excessive E2E coverage, and same-scope duplicates. A criterion tag whose assertion does not exercise the criterion is a hollow oracle.
- Treat every escaped defect as evidence that the portfolio missed a risk, technique, oracle, fidelity, or placement decision.
- Treat flakes, order dependence, uncontrolled time, shared state, and environment leakage as test defects.
- Use coverage as a gap locator. Use mutation testing selectively on changed or high-risk deterministic logic when assertion strength is uncertain.
- Remove a redundant broader test only after the smaller replacement proves the same scope; keep distinct broader backstops.
A quarantine must name an owner, reason, expiry, and retained signal. A quarantined test cannot count as verified evidence.
9. Hand off
- Hand selected checks to
defect-shift-leftonly after a Portfolio or Combined pass finalizes their architecture-dependent fields. - Send pipeline execution triggers, schedules, blocking behavior, artifacts,
freshness, and failure actions to
ci-cd-reliability-architecture. - Send criterion mappings and executed results to
requirements-traceability. - Send suite latency, resource cost, flakiness, queues, and duplication to
system-optimization. - Send a gate aggregator that could count skipped, retried, or quarantined
results as verified to
gate-lifecycle. - Send security, accessibility, safety, UX, data, or domain-specific obligations to the applicable companion skill without weakening them.
Decision Rules
| Condition | Decision |
| Every material risk has a credible oracle and sufficient evidence path | ADEQUATE |
| The critical path is covered but named residual gaps remain | PARTIAL |
| A material risk has no credible evidence path, or the portfolio gives false confidence | NOT-ADEQUATE |
| Meaning, oracle, environment, specialist policy, or owner is unresolved | DEFER |
An Obligation pass is provisional by design and cannot emit ADEQUATE. Reserve
ADEQUATE for a Portfolio or Combined pass against accepted architecture.
Do not emit ADEQUATE from test counts, pyramid shape, coverage percentage,
green CI, or one successful release.
Output Contract
Emit one row per material obligation:
| Risk / criterion | Failure mode | Oracle | Technique | Scope / fidelity | Dependencies | Data / environment | Evidence | Residual blind spot |
Then emit:
Subject: <requirement, increment, subsystem, change, or system>
Mode: Design | Audit
Strategy pass: Obligation | Portfolio | Combined
Decision: ADEQUATE | PARTIAL | NOT-ADEQUATE | DEFER
Test basis: <requirements, criteria, contracts, incidents, signals>
Architecture basis: <not yet fixed | accepted decision refs | stable existing>
Quality risks: <ranked risks or none found>
Manual evaluation: <exploratory, usability, accessibility, acceptance, none>
Adequacy signals: <trace coverage, gaps, mutation, escaped defects, flakes>
Handoffs: <shift-left, CI/CD, traceability, optimization, companions>
Residual risk: <accepted, blocked, deferred, unknown, or none identified>
Next action: <one concrete evidence, decision, or removal step>
Verification: <portfolio/test review, commands, results, or Not run + reason>Guardrails
- Do not invent requirement meaning, thresholds, users, traffic, or risk acceptance.
- Do not use a test framework preference as a strategy.
- Do not require every technique for every system.
- Do not equate mocks with isolation or real dependencies with confidence.
- Do not automate subjective evaluation merely to make it countable.
- Do not retain expensive tests because they were expensive to create.
- Do not claim an ownership boundary listed under Boundary; hand it off.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

