Failure classes

Without a name for the MECHANISM, the same mistake recurs under different shapes and nobody notices. One real session hit PROOF-INVALID five times in five shapes before it had a name. Name the mechanism, not the symptom.

How to use: match the incident to a class below; check whether it is the second case of that class (then stop and build a guard); apply the guard. A guard is something that runs (assert, lint, test, dead-man check, a marker string the next editor will see), never a note.

Measurement and evidence

• PROOF-INVALID — an invalid measurement still graded as a pass; a conclusion drawn from empty data. Guard: the measurement must prove it is valid (non-empty, covers the population, would have gone red) before it may conclude anything.
• WRONG-PROXY — measuring A and concluding B when A does not represent B (frame smoothness for CPU load; speech-recognition word timings for silence length). Guard: measure the physical quantity on the signal itself; use a model or proxy only to locate.
• GREEN-BUT-WRONG — the tool reports success but the real thing did not happen (is-active: active with nothing listening; a file-copy bridge reporting written while the target is unchanged; linter green on a broken file). Includes structured-output distortion: enum stalls, booleans flipping on caveats, answers flickering with input resampling. Guard: verify the effect, not the report (checksum both ends, connect to the port, re-read the file); write retries to a new path; for model output prefer free text + code parsing + a safe fallback + agreement of two asks.
• GUARD-NEVER-RAN — a guard did not run, but the dashboard lumps "did not run" with "ran and failed" (e.g. CI jobs ending in 3 s with 0 steps because billing failed), so everyone believes they are being watched. Signs: abnormally short run, zero steps, red with no test output, a new guard never seen firing. Guard: read evidence the check ran (for GitHub Actions: gh api repos/<o>/<r>/actions/runs/<id>/jobs -q '.jobs[] | "\(.conclusion) steps=\(.steps|length)"' plus the job annotations); report three states: green, red, did-not-run; give every scheduled guard a dead-man alarm placed on the node being operated.
• PROBE-ANSWERED-BY-HOP — a probe to the target is answered by an intermediary (CDN edge, tunnel, load balancer, test harness) or is smaller than the real payload, so it passes while the real path is broken (ping and small TLS handshakes pass; a 1.7 KB post-quantum ClientHello dies in a PMTU black hole). Guard: go all the way to the real target with an application-level transaction of realistic size; imitate the failing client (packet size, TLS groups, resolver); when one app fails and another works, list the stack differences and test each. Suspect MTU → try a smaller MTU first (one-command undo).
• LATEST-VERSION-ONLY — reading only the newest version of a document and concluding "X does not exist", while older or already-submitted versions contain it. Guard: claim absence only after a machine scan of every version and every copy sent out, stating how many files were scanned; when versions disagree, show both and ask.
• FALSE-ALARM-DRIVES-WRONG-ACTION — an alarm or count that does not subtract background noise drives an irreversible action (21 "lost" lines → git reset --hard of a correct cherry-pick; real loss was 0). Guard: before an irreversible action based on a count, state what the count is when everything is RIGHT and subtract it; scope bare identifiers before comparing; compare counts with the same script on both points in the same session.
• HAPPY-PATH-ONLY — done after trying only the success path. Guard: generate and run edge cases: empty/bad input, network failure, concurrency, scale, stale data, restart, permissions/secrets.

Building on assumptions

• BUILD-BEFORE-UNDERSTAND — building an artifact before MEASURING its foundational assumptions. Signs: writing code for a system you have not queried once; carrying knowledge from a familiar system to a variant (MySQL → Galera); "it probably works like X"; the third fix round on one artifact. Guard — assumption ledger: before the first line, list what MUST be true for the artifact to work, each with the command that proves it; measure all, then build; write unmeasurable assumptions into the artifact as unmeasured.
• SOLVING-WRONG-PROBLEM — an elaborate solution to dodge a cost the user was willing to pay. Signs: the constraint you optimise is self-imposed; the user said "confusing" more than once; you are mitigating risks your own solution created; a standard procedure already exists.
• LUCKY-PRECEDENT — a rule extracted from a case that worked thanks to something pre-existing you could not see; the written rule is half right until the next fresh case. Guard: write rules from the FULL STATE of the working case (dump it), not from the DELTA you typed; ask "what does the example already have that a new one will not?"; every template ends with a runnable acceptance check.
• DOC-VS-REALITY — the documentation or instructions say one thing, the system does another. Guard: check the running system; fix whichever side is wrong in the same change.

Ordering, scope and reach

• FIRST-MATCH-WINS — rules processed in order; a rule added later never matches (ufw, sshd 00-*.conf, iptables, routing tables). Guard: inspect the effective order and test the new rule's hit.
• SELF-LOCKOUT — a cut or tightening touches the path you are using to get in. Guard: keep a parallel or console path; change additively; verify the management path after each step.
• SCOPE-BLAST — a script or command changes more than its intended radius. Guard: dry-run listing every target first; explicit allow-list.
• WIDEST-READING — "remove X" where X exists at several layers (app setting, repo code, software on the user's machine, cloud resource), executed at the widest layer. Guard: before removing, write down the layers where X exists; act only at the layer being discussed; ask before touching the user's machine, account or outside data.
• PATCH-MATCHES-ELSEWHERE — an unanchored regex patch matches a different structure with the same key first (lazy .*? swallowing neighbouring keys), then a destructive step runs on the corrupted result. A match count of 1 does not help when the first match is the wrong one. Guard: replace whole printed lines exactly (assert count == 1) or anchor to the block; re-print or load the affected definitions before any delete/recreate step.
• TEMPLATE-AUTORUN — a template file with placeholders is picked up and executed by a pipeline as real code. Guard: templates carry a .template suffix and the pipeline has a content guard.

Errors and outcomes

• SWALLOWED-ERROR — an error falls into a silent except, 2>/dev/null or empty fallback (wrongness spreads quietly), OR nothing is caught and one bad item takes the whole system down. Guard: every handler logs a reason and counts; per-item isolation in batches; timeout, exit code and result reported as three separate facts.
• IDENTITY-CHANGES-AT-HOP — through an intermediary (proxy, WAF, NAT, LB, VPN, sidecar) the far end no longer sees your address or identity; the real error at the far hop is swallowed and replaced by an unrelated connection error (auth failure 1045 surfacing as "server has gone away"). Guard: a connection error that does not mention permissions → read the intermediary's and backend's logs FIRST; do not tune timeouts.
• ALERT-PER-SYMPTOM — one alert per symptom with no cause relationships plus repeat escalation: one root cause, 17 messages in 2 hours. The opposite reflex (switching off a noisy alert) is also wrong. Guard: cause→effect table and dependencies so one root gives one message; aggregate alerts for many observers of one target; estimate message count for a typical incident before enabling; severity by real impact; schedules and two-step thresholds instead of disabling; a periodic digest as the channel's own heartbeat; alert only on what you own; match service and on-call hours; recovery hysteresis.

Privilege and data exposure

• PRIVILEGE-LAUNDERING — an intermediary (stored procedure with definer rights, service account, sudo wrapper, CI runner, API gateway, a forwarded SSH agent or any lent credential) acts with its OWNER's rights, silently voiding the caller-side restriction. No error ever surfaces. Guard — two scope columns: before lending a credential or building an owner-rights carrier, write "scope NEEDED" next to "scope GRANTED" and compare; ask who owns it, who can touch it, how long the window stays open and how often; privileged bodies never take commands from outside (bind parameters, no string concatenation); prove the limit on the GRANTING side by attempting the forbidden action and reading the refusal (e.g. a read-only deploy key instead of agent forwarding).
• SIDE-CHANNEL-LEAK — data locked down on the main path leaks through a side channel without access control: logs, metrics, traces, error messages, backups, temp files, headers (statement logging that prints bound parameter values next to encrypted columns). Guard: when enabling any side channel, write into its config: what goes out, who can read it, how long it is kept; keep structure, drop values; a test that asserts the setting in every config file; keep a written list of side channels.
• GREP-PRINTS-SECRET-ROW — a search for one value prints the whole matching table row, including its password column. Guard: any script that prints table content masks cells by column header (password, secret, token, key) and prints only trimmed context; files named like credentials print only line numbers and labels.
• WIDEN-PROD-FOR-TEST — a test needs to clean up, and the nearest fix is granting the production role the right to do it. Guard: clean up with the role that already has that right by design (migration/owner role); if none does, the test is creating a state it should not — fix the test, not the schema. Prefer putting a VALID value into shared state over an invalid one you promise to clean.
• BACKUP-FILE-IN-WEBROOT — backing up code by renaming in place inside the webroot; the server returns source as plain text (.php.bak → HTTP 200). Guard: back up outside the webroot; block .bak/.old/.orig/.save/.swp/.tmp at the web server; verify with real HTTP both the block and the allow.

Deploy and environments

• HALF-DEPLOYED — a multi-node cluster deployed minus one node; every check is green because you measure the machine you touched; users see "works sometimes". Guard: deploy from a node inventory file; compare checksums across nodes of the cluster.
• DEPLOY-TREE-AS-WORKTREE — one directory is both where you try things and where production reads from; sooner or later production runs what you were trying. Silent while both branches happen to match. Guard: separate trees; a function at the start of the deploy path refusing to run unless the deploy tree is on the main branch, equal to origin and clean; a post-checkout hook alarm; a test that builds a real git tree and runs that function.
• SCHEMA-CODE-DRIFT — code and schema drift apart, both directions fail silently (a merge without its .sql misses a column; a dropped table kills a procedure outside the module). Guard: a merge without schema files is presumed to carry schema debt; check DB objects on every target before deploy, with a control that must fail when the check is broken; find every reader before changing or dropping a structure.
• NEW-WRITER-OLD-READER — a new write path produces rows (or a file format) the old readers consider impossible (null health column → TypeError, page 500; a ledger switched to 7-char SHAs makes update scripts exit silently for a week). Guard: when adding a writer, grep every reader; test with the MINIMAL row the new writer creates and open every reading screen; when changing a file format, run every script that reads it.
• EDIT-UNDER-RUNNING-OWNER — editing on disk a file whose running owner keeps a copy in memory and rewrites the whole file on state changes; the edit looks right, then silently vanishes minutes later (GUI/Electron app configs). Guard: quit the owner first or edit through its UI; check again AFTER the owner's next write (mtime change); a periodic audit as the last net. Detect the process by full path, not a name match that misses.
• GUARD-STARVES-HOST — a protective layer (snapshots, audit logs, WAL archive, versioning) shares a finite resource with what it protects, enabled with default policy and sized by the visible number instead of the real budget; under load it fills the pool and the host locks itself read-only. Guard: before enabling, write down the shared resource, the real budget measured at the allocation layer, and the consumption rate → derive retention; alert below the self-lock point; health-check a backup target by test-WRITING, not by mount status.
• CHECK-NOT-IN-PIPELINE — a checker exists and the rule "check after writing" is documented, but the check is a separate manual command; under time pressure it is skipped and the product ships unchecked. Guard: the script that produces the artifact calls the checker and exits non-zero on errors; a new rule is not done until it has an automatic place to run (hook, build script, CI).

Guards that lie

• GUARD-SCANS-PROSE — a "this string must not appear" guard scans whole files, so it hits comments and prose; the tempting fix is deleting the explanation. Guard: decide when writing the guard whether it scans code or prose and say so in the guard; strip comments before matching code; include control cases for both a violation and harmless prose.
• BLESSED-REGRESSION — refreshing a snapshot/golden file from current output and treating the next green as proof; a regression becomes the new standard. Guard: every snapshot suite has at least one semantic assertion not read from the expected file; expected-file diffs are reviewed like code; additions to exception lists carry a date and reason, and an audit counts them and asks for fewer.
• READONLY-BY-PROSE — a safety constraint handed to another agent only in words ("read only, do not run repo scripts") while its tools can run anything and foreign code sits on disk. Guard: build the constraint into the environment (e.g. clone with --no-checkout and read via git show HEAD:<path>; see vetting-third-party/scripts/vet_clone.sh); ask "what in the environment prevents this?" and build it if nothing does. A suspicious repo is vetted in a throwaway VM.
• FOLLOW-LINK-DELETE — a recursive delete given a symlink or junction follows it and deletes the real target. Guard: [ -L "$p" ] && rm "$p" before any recursive delete, or refuse; cleanup scripts get a test that links to a decoy directory which must survive.

Agents, models and supply chain

• FRAME-BLIND — a system prompt that frames the task makes the model blind to questions outside the frame (with the system prompt it denies a second document in an image; without it, it sees it clearly). Guard: diagnostic questions run WITHOUT the framing prompt, one question at a time; compound questions dilute and miss.
• FORCED-CHOICE-NO-ABSTAIN — a classifier (model, enum in an output schema, choice function) has no "unknown" option, so out-of-scope input still gets a label with high confidence; the probability is a forced distribution, not evidence. Guard: every classification has an other/unknown option and a review threshold (auto-accept only above a calibrated probability and margin, otherwise a review queue); a control test where an out-of-class sample must come out other; report counts of auto vs review.
• REMOTE-INSTRUCTIONS-AT-RUNTIME — a skill or tool trusted as "always loaded" whose real procedure lives at an external URL fetched at run time (llms.txt, live docs "as source of truth"): whoever controls the URL controls the agent, and two runs differ with no repo diff. Often paired with a broad description without a negative clause. Guard: reject skills whose SKILL.md reads/fetches/follows URLs as procedure; copy needed reference into references/ with the source SHA; vet external skills first; descriptions carry a negative clause.
• ONBOARDING-OVER-REVERSIBILITY — a "one command and you're done" installer optimised for minute one: secrets written into config files, files overwritten without backup, unpinned dependencies (npx -y, uvx, curl | sh), no uninstall. The cost shows at every later start. Guard: never run third-party installers on the real machine; write the config by hand, pin @<version>, keys via environment variables from a keychain; run vetting-third-party/scripts/check_mcp_config.py after any MCP change.

Adding a class

Name the mechanism: the name must answer "what makes this mistake repeat", not "what it looks like". Add an entry with signs, at least one real dated case (keep private details out of public copies), and a runnable guard. Then add the name to the class list in AGENTS.md (scripts/check.sh fails until both lists match). When you fix a case, leave a marker string in the fixed file (a distinctive comment that must stay while the fix stays) and a periodic check for it: a vanished marker means the fix was silently reverted. Record unresolved cases too, with the reason.

Original taxonomy by the repository author, generalised from real incidents (private case details removed). FOLLOW-LINK-DELETE and BLESSED-REGRESSION distilled from deepseek-ai/deepseek-harness (MIT); marker-string idea from ruvnet/ruflo witness (MIT, idea only). See SOURCES.md.