agentleFS
markoblogomarkoblogo/abvx-agent-skills/skills/bug-evidence-protocol/SKILL.md

bug-evidence-protocol

Skill16 starsChanged 5 months agoMITView on GitHub

Capture and classify minimal red-to-green evidence for a diagnosed bug and scoped fix, with command hashes, Git/environment identity, broader-check records, risk-based approval, route evidence, and optional cpat linkage. Use after diagnose establishes a credible reproducer or when a reported regression needs an auditable proof packet; do not use as an autonomous bug scanner.

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 4 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈4

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Do not scan a repository for unsolicited bugs or claim a defect from inspection alone.

    line 19
  2. Distinctive:

    Stop for explicit approval before:

    line 25
  3. Distinctive:

    Never capture a full environment dump.

    line 46
  4. Very specific:

    Do not accept a prose or CLI flag such as full suite passed as a substitute for captured check records.

    line 67
  5. Very specific:

    Never promote route accepted to used and confirmed from agent prose.

    line 86
  6. Distinctive:

    Keep raw bulky or protected output in its authoritative evidence store.

    line 96

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Bug Evidence Protocol

Record proof without replacing the workflow that produces it:

  • diagnose owns reproduction, hypotheses, and root cause;
  • test-driven-execution owns the behavior-first red/green implementation loop;
  • this skill captures and classifies evidence;
  • git-native-context-contract may retain a recurrent lesson as a cpat.

Do not scan a repository for unsolicited bugs or claim a defect from inspection alone.

Authority and approval

Invocation does not reduce authority already granted by the user. Reversible local tests and scoped code fixes may proceed when they are part of the requested task.

Stop for explicit approval before:

  • production or device writes;
  • destructive migrations or data mutation;
  • dependency or configuration changes outside the approved scope;
  • crossing security, privacy, financial, or protected-evidence boundaries;
  • expanding files, behavior, or external effects materially beyond the request;
  • changing production code during an audit-only or reproduce-only task.

Evidence integrity

A confirmed bug requires a valid reachable path, a defensible expected behavior, and a focused command that fails for the predicted symptom. Syntax, fixture, dependency, permission, and unrelated setup failures are INCONCLUSIVE, not REPRODUCED.

Capture the exact targeted command before and after. Preserve argv rather than a shell string. The included capture helper records:

  • SHA-256 command identity;
  • cwd, runtime, Git root/head/dirty state;
  • hashes of selected lockfiles when present;
  • time, duration, exit code, and timeout;
  • bounded redacted stdout and stderr.

Never capture a full environment dump. Inspect captured artifacts before publishing them; regex redaction is a safety net, not proof that output is public-safe.

python3 <skill>/scripts/capture_command.py --label targeted-before --output before.json -- <exact command>
python3 <skill>/scripts/capture_command.py --label targeted-after --output after.json -- <exact command>
python3 <skill>/scripts/capture_command.py --label broader-tests --output broader.json -- <relevant suite>

Classification

Use only:

  • REPRODUCED: the targeted before record fails for the confirmed symptom.
  • NOT_REPRODUCED: the proposed reproducer passes before a fix.
  • NO_BUG_PROVEN: investigation produced no credible failing case.
  • INCONCLUSIVE: setup, environment, nondeterminism, command mismatch, or symptom ambiguity blocks a conclusion.
  • STILL_FAILING: the same targeted command still fails after the attempted fix.
  • FIX_UNVERIFIED: targeted evidence turns green but captured broader evidence is absent.
  • FIX_REGRESSION: targeted evidence turns green but a captured relevant broader check fails.
  • FIX_PROVEN: the same command hash turns red to green, symptom match is confirmed, and all captured relevant broader checks pass.

Do not accept a prose or CLI flag such as full suite passed as a substitute for captured check records.

python3 <skill>/scripts/classify_evidence.py \
  before.json after.json result.json \
  --symptom-match confirmed \
  --broader broader.json \
  --route-state unavailable \
  --route-detail "no routed executor used"

Route evidence

Record exactly one state:

  • route accepted: route controls were accepted, but effective runtime identity is not proven;
  • used and confirmed: host metadata confirms the effective runtime or route;
  • unavailable: routing failed, is absent, or was not relevant; include a reason.

Never promote route accepted to used and confirmed from agent prose.

CPAT handoff

For a recurrent incident, CI/runtime defect, or device repair, create or update a cpat only after verification. Its Verification section should reference the stable evidence ID and summarize:

symptom -> reproduction evidence -> root cause -> change -> scope -> red/green verification -> prevention

Keep raw bulky or protected output in its authoritative evidence store.

Final report

Lead with the status and include evidence ID, targeted command identity, before/after exit codes, broader checks, Git SHAs, route state, limitations, residual risk, and cpat_id when present.

Attribution

Adapted from Kappaemme-git/codex-bug-reproducer: minimal reproduction, honest evidence labels, same-command red-to-green proof, and structured reporting. ABVX changes replace unconditional two-gate consent with risk-based approval, remove autonomous bug hunting, require captured broader-check records, add Git/environment/command identity, preserve route states, and link recurrent lessons to cpat.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.