writing-tests
Skill0 starsChanged 8 months agoApache-2.0View on GitHub
Test writing standards — formula-derived bounds, tests must fail to reveal, mock discipline, contract tests. Use when creating or editing test/ files.
42 other projects use a version of this.
Adopted by
Most recently taken up by
- chrisezedi/claude-skillstheir own version, first seen Nov 16, 2026
- RMATI-11861/Claude-Skillstheir own version, first seen Nov 16, 2026
- ZaxbyHub/opencode-swarmtheir own version, first seen Nov 15, 2026
- iliaal/ai-skillstheir own version, first seen Nov 14, 2026
- PostHog/posthogtheir own version, first seen Nov 14, 2026
- PostHog/posthog-fosstheir own version, first seen Nov 14, 2026
Used with
Projects with it also use reviewing-code, commits and 5 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
bostonaholicSkillreviewing-code
Code review methodology for evaluating implementation changes. Use when reviewing code changes for quality, design, correctness, and maintainability. Focuses on changes made during implementation using Conventional Comments for clear, actionable feedback.
Also in 4 projects that use this
JetBrainsSkillcommits
Use this skill whenever the user asks to commit changes, write or fix a commit message, amend or rename a commit, or do a workflow that includes committing in the IntelliJ repository. This is a thin repo-specific overlay: use IntelliJ commit format, write full commit messages by default, and keep requested suffixes such as IJ-MR trailers in a final separate paragraph.
Also in 3 projects that use this
OpenAISkillrunning-tests
Guide for running tests in the openai-dotnet repository. Use this when asked to run, debug, or validate tests, or when writing new tests. Explains test modes (Playback, Record, Live), how to identify recorded vs non-recorded tests, environment variable configuration, and what to do when recordings are missing or stale.
Also in 3 projects that use this
BlockSkilldebugging
Debugging guidelines and best practices
Also in 5 projects that use this
VercelSkilltechnical-writing
Write, edit, review, or audit user-facing documentation for the eve repository. Use for changes under docs/, documentation tied to eve APIs or CLI behavior, docs work based on Slack or support feedback, and requests to make eve docs clearer, more natural, or less AI-patterned while verifying claims against current source, tests, CLI help, public releases, and repository conventions.
Also in 3 projects that use this
SentrySkillsecurity-review
Finds exploitable application security vulnerabilities in code changes. Use for Warden security scans, appsec review, OWASP-style checks, authentication or authorization bugs, injection, XSS, SSRF, path traversal, secrets, unsafe crypto, webhook verification, open redirects, or sensitive data exposure.
Also in 3 projects that use this
Other versions42 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 42
- remotion-dev/remotion★ 73ksame name
- PostHog/posthog★ 41ksame name
- JetBrains/intellij-community★ 21ksame name
- quarkusio/quarkus★ 16ksame name
- mcp-use/mcp-use★ 11ksame name
- DetachHead/rebased★ 6.5ksame name
- chattocorp/chatto★ 2.9ksame name
- starkware-libs/cairo★ 1.9ksame name
- bholmesdev/hubble.md★ 1.5ksame name
- amplifthq/openmeld★ 1.4ksame name
32 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository12 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- specification/CLAUDE.mdCLAUDE.md
- src/CLAUDE.mdCLAUDE.md
- tasks/CLAUDE.mdCLAUDE.md
- test/CLAUDE.mdCLAUDE.md
- cleanupSkill
- diagnoseSkill
- sddSkill
- triageSkill
- writing-architectureSkill
- writing-specsSkill
- writing-tasksSkill
The file
We split this skill into the rules it gives an agent
6 rules. 0 are common, 3 are distinctive, and 3 are very specific to this project.
0
Common
Most of the 42 edited versions we read keep these.
3
Distinctive
This author's own choices. Worth a look.
3
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 10
When a specification defines a formula, all test bounds and expected values must be computed from that formula -- never estimated by …
- Distinctive: line 33
No conditional verification: If a contract requires fields to be populated, assert they are populated unconditionally.
- Distinctive: line 34
Prefer a failing test over a passing one: When an implementation bug causes a test to pass for the wrong reason (e.g., empty output …
- Very specific: line 39
Every Mock() or patch() requires a corresponding contract test that exercises the real implementation against the same interface.
- Very specific: line 59
Before declaring a task phase complete, verify every Mock()/patch() has a corresponding contract test.
- Very specific: line 67
When a test appears to conflict with its specification, file feedback in test/feedback/<test-file-name>.md describing the discrepancy.
“Common” is measured: kept by at least 6 in 10 of the 42 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Test Writing Standards
Numeric Bounds Must Be Formula-Derived
When a specification defines a formula, all test bounds and expected values must be computed from that formula -- never estimated by intuition.
- Compute the expected value by substituting the test input into the spec formula before choosing an assertion bound.
- Show the derivation in a comment next to the assertion so reviewers can verify it.
- Do not use "round number" bounds (e.g.,
< 1.01) unless the formula confirms they hold at the chosen input.
Example -- wrong:
# "Should be very close to 1.0 for large freq"
assert symbol_weight(1_000_000) < 1.01 # intuition, not derivedExample -- correct:
# 1.0 + 2.0 / log2(1_000_001) ~ 1.1003
assert symbol_weight(1_000_000) < 1.2Tests Must Fail to Reveal, Not Pass to Reassure
A test that cannot fail is worthless. Every assertion must be capable of catching a real deficiency in the implementation. A passing test should mean the implementation satisfies the spec -- not that the test avoided checking.
- No vacuous assertions.
assert isinstance(result, list)followed byif result: assert ...verifies nothing when the implementation returns[]due to a bug. If the spec says the list should be populated, assertlen(result) > 0. - No conditional verification. If a contract requires fields to be populated, assert they are populated unconditionally. A test that only checks field values "if they exist" will pass when the implementation silently returns empty data.
- Prefer a failing test over a passing one. When an implementation bug causes a test to pass for the wrong reason (e.g., empty output triggers the correct error path by accident), rewrite the test so it fails until the bug is fixed.
- Contract tests must exercise the real interface. A contract test that gets no data from the real backend (due to a transport bug, timeout, etc.) and passes anyway is not a contract test -- it is dead code with a green checkmark.
Mock Discipline
Every Mock() or patch() requires a corresponding contract test that exercises the real implementation against the same interface. Skipping via pytest marker (e.g., @pytest.mark.requires_coq) is acceptable when external tools are needed; omitting the test is not.
# Good: consumer test mocks the backend
def test_pipeline_calls_backend():
backend = Mock()
backend.list_declarations.return_value = [("A", "Lemma", {})]
...
# Good: contract test verifies real backend satisfies the same interface
@pytest.mark.requires_coq
def test_coq_lsp_backend_list_declarations():
backend = CoqLspBackend(...)
decls = backend.list_declarations(Path("test_fixture.vo"))
assert isinstance(decls, list)
assert all(len(d) == 3 for d in decls)
# Bad: consumer test exists but no contract test for the real implementationBefore declaring a task phase complete, verify every Mock()/patch() has a corresponding contract test. If not, the phase is incomplete.
Mock Return Values Must Use Real Types
Mock return_value must use the actual type the real implementation returns (e.g., dataclass, not dict). This exercises serialization and attribute-access paths in the consumer.
Test File Feedback
When a test appears to conflict with its specification, file feedback in test/feedback/<test-file-name>.md describing the discrepancy. Do not silently adjust the test or the implementation.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

