slopproof / tests
AGENTS.md2 starsChanged 20 days agoMITView on GitHub
The test manual. Read it before you write or change a test. The root AGENTS.md still applies. Test each behavior in the cheapest layer that can observe it: Every new function is reached by a test in the cheapest layer that can observe it, and every bug fix adds a test that fails without the fix. Fakes are named and defined once: a Layer in tests/api/harness.ts, a handler factory in tests/component/api-mocks.ts, a module in tests/component/stubs/, a helper like captureLog (tests/unit/process-fakes.ts);…
Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.
Of the AGENTS.md files we read, the 31,542 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.
In this repository10 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- tests/CLAUDE.mdCLAUDE.md
- add-featureSkill
- add-pageSkill
- api-changeSkill
- auth-changeSkill
- database-changeSkill
- shadcnSkill
- upgrade-prerelease-depsSkill
The file
We split this AGENTS.md into the rules it gives an agent
3 rules. About 0 are common, 2 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 5
Test each behavior in the cheapest layer that can observe it:
- Distinctive: line 16
Every new function is reached by a test in the cheapest layer that can observe it, and every bug fix adds a test that fails without the fix.
- Very specific: line 60
Contract coverage: After a full verify:app, scripts/contract-coverage.ts checks that every operation × status in openapi.json was answered …
≈ “Common” is an estimate: at least 1 in 5 of the 40,246 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
tests/AGENTS.md
The test manual. Read it before you write or change a test. The root AGENTS.md still applies.
Test each behavior in the cheapest layer that can observe it:
| Layer | Where | Run by | For |
| unit | tests/unit | check | Pure functions, examples plus fast-check properties. Modules in COVERAGE_GATE (vitest.config.ts) need 100% lines and branches |
| api | tests/api | check | Effect handler branching through the typed client: in-memory PostsRepo, fake session store, no database |
| component | tests/component | check | React components in Chromium (Vitest browser mode), network mocked by MSW: pending and disabled states, every error branch, limits, focus |
| db | tests/db | test:db, verify:app | Server modules on a real Postgres without a build: the rate-limit storage, and the query budget of every repository method |
| integration | tests/integration | test, verify:app | The built app over HTTP: SQL, Better Auth, CSRF, headers, rate limits, shutdown |
| e2e | tests/e2e | test:e2e, verify:app | Browser flows, axe on every page state, keyboard and focus, ARIA landmark snapshots |
Every new function is reached by a test in the cheapest layer that can observe it, and every bug fix adds a test
that fails without the fix. Fakes are named and defined once: a Layer in tests/api/harness.ts, a handler factory
in tests/component/api-mocks.ts, a module in tests/component/stubs/, a helper like captureLog
(tests/unit/process-fakes.ts); a test body does not stub a project module itself. A test body holds at most 15
statements (Oxlint max-statements; a describe has no length limit) and four nested callbacks.
Running
pnpm test:unit|test:api|test:component [filter ...]runs one fast layer;pnpm test:fastruns all three with coverage.pnpm test:db [filter ...]runs thedblayer against a freshapp_db_<pid>_testdatabase next toDATABASE_URL(dropped afterwards;KEEP_TEST_DB=1keeps it). It needs Postgres, no build.pnpm test [filter ...](Vitest projectintegration) andpnpm test:e2e [filter ...](Playwright) each start the built app themselves, afterpnpm build, with Postgres and Mailpit (pnpm mail:up) running. Set no variables by hand: the runner's global setup (tests/integration/global-setup.ts,tests/e2e/global-setup.ts, both throughstartTestServersinscripts/test-servers.ts) refuses a stale.output, creates a freshapp_<runner>_<pid>_testdatabase, starts the two servers described below with two verified authors, and stops them and drops the database at the end. App logs go totest-results/app-server-<runner>*.log. The same holds forpnpm exec playwright test --uiand the VS Code Playwright and Vitest extensions.pnpm build && pnpm verify:appruns, in order and each even after a failure, the api layer with its recorder, thedblayer, the integration tests, the E2E tests (projects fromPW_PROJECTS) and the contract-coverage check, then prints a summary. It takes no filter: for one layer or file, run that runner directly. The servers start once per runner.- Install the test browsers once:
pnpm exec playwright install chromium firefox.verify:app,lighthouseand thecomponentproject useCHROME_PATHinstead of Playwright's Chromium when it is set. E2E runs every flow on the Playwright projectschromium,firefox,webkit,Pixel 7andiPhone 15; outside CI the default ischromium,firefox, because WebKit needs Ubuntu's libraries.PW_PROJECTS=all(or a list) chooses;pnpm ci:local verifyruns all five. - For a failed E2E run, open the trace:
pnpm exec playwright trace open test-results/<test>/trace.zip(see docs/agents/skills.md).
Rules per layer
Each layer's harness and rules are in its folder: api, db, component, e2e, and integration, which also describes the running app both integration and E2E test. Claude Code loads the one for the folder you open.
- Coverage.
pnpm test:fastwritescoverage/index.htmlfor all ofsrcand fails unless every module inCOVERAGE_GATEis fully covered (lines and branches) and the whole ofsrcstays aboveCOVERAGE_FLOOR, set a little under the measured totals. Add security-critical and fully tested modules to the gate with their tests; raise the floor when coverage grows. - Properties. Parsers, validators and anything security-relevant get fast-check properties next to their
examples (
import * as fc from 'fast-check'): round trips, arbitrary input never throws, invariants such as "the TCP peer is the last hop" or "a returned redirect is accepted as it is". When a property finds a case, add it to the examples too, so the failure stays named. - Contract coverage. After a full
verify:app,scripts/contract-coverage.tschecks that every operation × status inopenapi.jsonwas answered by some test (the app servers' request logs, plus the api layer through the harness's recorder,CONTRACT_OBSERVATIONS) and that no operation answered a status it does not declare. A new declared status needs a test that provokes it; a gap that cannot be tested goes intests/contract-coverage-allowlist.jsonwith its reason (today: the CSRF 403 and the empty 500 of a defect, which the API description documents once). Stale entries fail. - Flakiness. CI retries a failed Playwright test once but fails the run if it then passes
(
failOnFlakyTests): fix the cause. Locate fields by role and exact name (getByRole('textbox', { name: 'Email', exact: true })); a baregetByLabel('Email')also matches "Email confirmed". Mutation testing is not a gate yet: StrykerJS's Vitest runner skips nested tests on Vitest 5 (docs/stack.md).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

