prove-it-can-fail
Skill27 starsChanged 18 days agoApache-2.0View on GitHub
Every test, seed, verification script, CI gate, or check must demonstrably be able to fail when the thing it guards is broken. Ask of each check "what does this print if the thing is broken?" Triggers when writing or reviewing any test, smoke, seed, verification file, or CI job, when marking a test skipped, and when arming a new gate. Does not fire on triaging an already-red suite (mass-red-triage) or on how tests touch shared data (safe-smokes).
In this repository56 other files it gives its agents.
- ai-surface-disciplineSkill
- blast-radius-grepSkill
- checkpoint-logSkill
- commission-the-rosterSkill
- committee-reviewSkill
- confirm-before-pushSkill
- constant-time-secret-compareSkill
- context-economySkill
- contract-reviewSkill
- cross-agent-reviewSkill
- date-parse-utc-safeSkill
- db-migration-verificationSkill
- deliverable-integritySkill
- dependency-upgrade-verificationSkill
- earn-every-lineSkill
- email-delivery-verificationSkill
- enforce-invariants-in-buildSkill
- env-change-verificationSkill
- findings-are-evidenceSkill
- fixtures-from-realitySkill
- flag-deferred-itemsSkill
- guard-the-spend-pathsSkill
- handoverSkill
- honest-failure-surfacingSkill
- legal-citation-integritySkill
- legal-notice-handlingSkill
- live-data-surgerySkill
- live-state-firstSkill
- lock-at-the-chokepointSkill
- mass-red-triageSkill
- measure-per-routeSkill
- no-silent-data-dropSkill
- one-real-rideSkill
- outbound-side-effect-idempotencySkill
- parallel-work-reconSkill
- pin-to-what-was-seenSkill
- plan-firstSkill
- price-the-spendSkill
- reproduce-the-real-buildSkill
- rerun-before-verdictSkill
- safe-smokesSkill
- secrets-in-outputSkill
- server-side-authoritySkill
- skill-library-builderSkill
- spawn-from-declared-stateSkill
- statute-to-obligations-registerSkill
- substantiate-outward-claimsSkill
- team-loopSkill
- tool-layer-bytesSkill
- trace-one-recordSkill
- unordered-eventsSkill
- unslop-codeSkill
- unslop-textSkill
- unslop-uiSkill
- verified-citationsSkill
- verify-the-effectSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 3 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 44
A passing test can encode the bug: When fixing a bug, check whether a green test pins the wrong behaviour — two tests once codified …
- Distinctive: line 46
Assert that the subject actually ran: When a test drives an external script, binary or process, a bail before any work makes every negative …
- Distinctive: line 60
Test, smoke, or seed touches a shared or live database → load safe-smokes before writing it.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Prove it can fail
A green board proves nothing unless every light on it can go red. This skill exists because nine-plus separate incidents produced green signals with no assertion behind them — including a statutory control that had never been asserted on any run, ever, and verification files that printed pass on any error. Layer: hub for test-and-check integrity (this is one of the six always-on norms; the norm line is the everyday trigger, this file is the playbook).
The question that does the work
For every check you write or review, ask: what does this output if the thing it guards is broken? If the honest answer is "still green", "a skip", or "nothing", the check is theatre. Answer it by demonstration where cheap: break the guarded thing once, watch the check go red, revert.
Rules
- A server reply is never a skip. Skips are for transport failures only — could not
connect, no credentials. A catch-all
catch { skip }hid both a never-passable gate test and a would-be-green total outage. A skipped regulated check is unverified, not green; say so in the run summary. - Seeds get assertions. Fire-and-forget seeding let an RLS isolation test pass for the wrong reason (nothing was seeded, so nothing leaked). Assert the seed landed before asserting the behaviour.
- Negative assertions need a positive control. "User B sees nothing" is only meaningful next to "User A sees the row". Without the positive control, an empty database passes.
- Only the specific expected error proves a control fired. "It threw" is not "the gate caught it" — assert the error identity (a stable code, not the human-readable message).
- A check whose two operands share a source is theatre. A "reconciliation check" that compared a value against its own definition could never fail. Trace both operands to independent sources.
- Watch a newly-armed gate's first run to completion — a nightly-gated spec once merged having never executed. And after fixing a pipeline's first red step, watch the whole run go green: a second failure can be queued invisibly behind the first.
- Test every mutation shape the app performs against a guard — set, swap, and null are three different attacks; a guard proven against one is unproven against the others.
- Assert on the subject, not a document-wide pattern. A broad "no X anywhere in the output" assertion is a hidden dependency on everything else and breaks on innocent code.
- A passing test can encode the bug. When fixing a bug, check whether a green test pins the wrong behaviour — two tests once codified drop-this-content as correct.
- Assert that the subject actually ran. When a test drives an external script, binary or process, a bail before any work makes every negative assertion trivially true. A launcher suite once exited at a pre-flight path check and reported nine failures beside four VACUOUS PASSES — "does not start a turn" and "the stranger is left alone" are both true of a process that did nothing — so the green ones read as proof that the guards worked. Guard the premise: assert the subject reached the behaviour under test, and treat a bail as a harness fault reported once, not as N results. Related to rule 2, but the failure is one step earlier: there the fixture was missing, here the subject never started.
Mechanics note: a CI check that reads a snapshotted event payload (e.g. a PR body) cannot be re-run into a pass after the source is edited — close/reopen refreshes the payload.
Routes
- Test, smoke, or seed touches a shared or live database → load safe-smokes before writing it.
- A suite has gone broadly red and you are tempted to debug → load mass-red-triage first.
What this skill does not do
It does not triage failures (mass-red-triage) and does not govern shared-data hygiene (safe-smokes). It governs whether a check is capable of telling you the truth.
Why
Green signals with no assertion behind them are worse than no check: they convert "unknown" into "verified" in every reader's head. Evidence: PropOS LESSONS_LEARNED 2026-07-14 (two entries), 2026-07-19 (two entries), 2026-07-15, Sessions 16, 18, 30, 33; ASH LESSONS_LEARNED 2026-06-09 and 2026-06-04.
Additions from the misses-log sweep (2026-10-22)
- A reference frozen when written alarms on history. A monitor judges "new" by each item's own timestamp in a rolling window, never against a count captured when it was written. (AI domain L-092.)
- A gate never compares against evidence a later step produces. Check, for each artefact a gate reads, that the run order creates it first. (passive income L-059.)
- Restore a mutation by reverse edit or a copy, never by checkout.
git checkout -- <file>after a red-proof also discards uncommitted work in that file and may rewrite its line endings. (AI domain L-145.)
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

