rust-in-peace
AGENTS.md25 starsChanged 4 months agoApache-2.0View on GitHub
This repo has two halves: - Interactive skills (.Codex/skills/) — read and write files in the repo (no target-code execution), run in this session: /quickstart (front door / Q&A), /threat-model (bootstrap, interview, or bootstrap-then-interview → THREAT_MODEL.md), /vuln-scan (static review → VULN-FINDINGS.json), /triage (verify + dedupe + rank a findings pile), /patch (generate candidate fixes → PATCHES/), /customize (port the pipeline to another stack). Route the user to these for scoping, static review, Q&A, and post-run triage. (/verify is contributor tooling…
Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.
Of the AGENTS.md files we read, the 32,671 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.
In this repository21 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- customizeSkill
- patchSkill
- quickstartSkill
- sast-drivenSkill
- sast-prioritiseSkill
- threat-modelSkill
- triageSkill
- variant-scanSkill
- verifySkill
- vuln-scanSkill
- customizeSkill
- patchSkill
- quickstartSkill
- sast-drivenSkill
- sast-prioritiseSkill
- threat-modelSkill
- triageSkill
- variant-scanSkill
- verifySkill
- vuln-scanSkill
The file
We split this AGENTS.md into the rules it gives an agent
1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 264
Model is a runtime arg, not config: Always --model flag or VULN_PIPELINE_MODEL env.
≈ “Common” is an estimate: at least 1 in 5 of the 41,581 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Codex for Securing Source Code
This repo has two halves:
- Interactive skills (
.Codex/skills/) — read and write files in the repo (no target-code execution), run in this session:/quickstart(front door / Q&A),/threat-model(bootstrap, interview, or bootstrap-then-interview →THREAT_MODEL.md),/vuln-scan(static review →VULN-FINDINGS.json),/triage(verify + dedupe + rank a findings pile),/patch(generate candidate fixes →PATCHES/),/customize(port the pipeline to another stack). Route the user to these for scoping, static review, Q&A, and post-run triage. (/verifyis contributor tooling for validating harness changes on docker-less hosts, not part of the user flow.) vuln-pipeline(harness/) — the autonomous pipeline. Docker + capability-routed detectors (rust: Miri/ASan/panic/hang + cargo-fuzz; cpp: ASan), executes target code, needs a sandbox (seedocs/security.md). Route here when the user wants to actually find and verify crashes.
Docs for each topic are in docs/; targets are in targets/ (canary is the
fast smoke test). The pipeline is profile-driven (profiles/): rust is the
primary profile — capabilities-routed detectors (capabilities.json), the
find→fuzz reattack bridge, the scorecard discipline, and the DVRA benchmark
(targets/dvra3-parser) — with cpp retained as the base profile. The rest of
this file is the pipeline operator guide.
vuln-pipeline
Execution-verified vulnerability discovery. The rust profile is the reason this
fork exists: a find-agent reads source, crafts inputs, and drives capability-routed
detectors (Miri / ASan / panic / hang, plus cargo-fuzz) until it lands a 3/3
reproducing crash. The retained cpp base runs the same loop over an
ASan-instrumented C/C++ binary. A grade-agent in a fresh container verifies it.
Output is a crashing input file, not prose.
When the user asks you to run it
Recommended: streaming mode with CC as the dashboard. Walk the user
through recon interactively (show the discovered focus areas, let them
adjust), then launch --runs N --parallel --stream in the background. Reports
land in reports/bug_NN/ as each crash passes grade and the judge clears it —
first one typically in minutes, not hours. Tail the reports directory and
surface each new report.json to the user with a summary as it appears.
Status check when asked: "N finds alive, M graded, K reports written — here's
the newest." Stragglers don't block disk writes; a stuck find-agent slows
itself, not the batch.
For first-time use on a target, run a small wave first (--runs 3 --parallel
--stream --max-turns 100) so the user gets a feel for token burn and prompt
calibration before scaling up. Check transcripts together if something looks
off.
Launch long runs in the background (tmux with a stable socket, or
run_in_background) so you can tail logs, watch found_bugs.jsonl, and
answer the user's questions without blocking.
Running it
Each pipeline phase is a standalone subcommand:
vuln-pipeline recon <target> --model <model> # propose focus_areas (YAML → stdout)
vuln-pipeline run <target> --model <model> # find + grade, one run (profile from config.yaml: rust or cpp)
vuln-pipeline run <target> --aggregate union # union-of-N cross-run combiner (default; recall-first)
vuln-pipeline run <target> --runs N --parallel # N concurrent finds, round-robin over focus_areas
vuln-pipeline run <target> --auto-focus # recon first, use its partition
vuln-pipeline run <target> --stream # judge + report stream in as grades land (recommended)
vuln-pipeline run <target> --find-only # skip grade (prompt iteration)
vuln-pipeline run <target> --accept-dos # DoS-class crashes count as valid finds (benchmark mode)
vuln-pipeline run <target> --resume <results_dir> # continue a partially-killed batch (skip terminal runs)
vuln-pipeline dedup results/<target>/<ts>/ # group crashes by signature (summary view)
vuln-pipeline report results/<target>/<ts>/ # batch-mode: exploitability analysis per unique crash
vuln-pipeline report results/<target>/<ts>/ --fresh # ignore existing bug_NN/report.json checkpoints
vuln-pipeline patch results/<target>/<ts>/ # generate + verify a fix per unique crash
vuln-pipeline patch results/<target>/<ts>/ --bug N --no-reattack # one bug, faster (skip re-attack tier)
vuln-pipeline reattack results/<target>/<ts>/ --aggregate union # find→fuzz bridge (rust): findings → reproducing fuzz harnesses
vuln-pipeline scorecard results/<target>/<ts>/ # gate on the reattack scorecard (rust)Results → results/<target>/<timestamp>/. For --runs N: subdirs run_000/,
run_001/, etc., plus a shared found_bugs.jsonl at the batch root.
Reports → results/<target>/<timestamp>/reports/bug_NN/ (manifest at
reports/manifest.jsonl).
Watching a run
Two output layers, both on by default:
- Heartbeat (stdout, every ~25 turns):
[agent] N tool calls (M msgs)— proves the process is alive during long silent stretches. - Progress (stderr, per-action):
[find:N] → Bash: /work/entry ...— shows what the agent is doing. One line per tool call and text block.
Transcripts stream to {find,grade,recon,report}_transcript.jsonl as messages
arrive (fsync'd). With --stream, judge transcripts go to
reports/judge_runNNN.jsonl and the report-grader's to
report_transcript_..._grader.jsonl. A mid-run kill leaves readable
transcripts on disk — nothing is buffered in memory waiting for the run to
finish.
Checking a background run: tail the log, cat found_bugs.jsonl for crashes
landed so far, count run_*/result.json files for graded results. Each
result.json writes the moment its run's grade finishes — stragglers don't
block disk writes. With --stream, also ls reports/bug_*/report.json for
reports landed, cat reports/manifest.jsonl for bug-id assignments, and
cat reports/judge_log.jsonl for the per-crash NEW/DUP_BETTER/DUP_SKIP
verdicts with reasoning. A stuck straggler can be killed (docker rm -f
find_<target>_<N>).
Rate limits / API errors don't kill runs. Each agent is one long-lived
Codex -p session; a 429 or 5xx mid-run is retried inside the CLI first,
and if the CLI gives up the pipeline backs off (exp, cap 300s) and relaunches
with --resume <session_id> — full conversation context restored, agent
continues from the turn that failed. Up to 20 resumes per agent
(agent.py:run_agent). Tell users they can run near their provisioned
capacity and let backoff absorb bursts; they don't need to self-throttle to
avoid losing work. The per-agent resume count is printed to stdout alongside
the [find:N] done ... summary line; it is not persisted to result.json.
Report phase
With --stream (recommended): reports are automatic. When each grade
lands, a judge agent decides whether the crash is a new bug, a cleaner
representative of an existing one (re-report), or a duplicate to skip. New
bugs trigger a report agent immediately; no dedup gate, no batch wait.
Reports land in reports/bug_NN/report.json minutes after the first
crash_found, not hours after the last straggler.
Standalone vuln-pipeline report <results_dir> stays for batch-mode
recovery or a results dir produced without --stream: one report agent per
unique crash signature, in a fresh container with only the PoC bytes and
source tree.
Output is a structured exploitability analysis — primitive characterization, reachability from the real attack surface, heap layout, escalation path, constraints — plus an agent-judged severity rating. A separate grader agent scores the report (semantic rubric, not keyword scan).
Before launching reports, ask whether the user wants the novelty check
enabled. --novelty makes the pipeline shallow-clone the target's
github_url on the orchestrator host and inject git log <commit>..HEAD --
<crash_file> into the prompt so the report can state FIXED/UNFIXED. Off by
default — air-gapped or outbound-restricted environments won't want the
pipeline reaching GitHub. The report container's egress stays restricted to
the API either way; only the orchestrator touches GitHub.
vuln-pipeline run <target> --stream --novelty --model <m> # streaming, with novelty
vuln-pipeline report results/<target>/<ts>/ --model <m> [--novelty] # batch-mode / recoveryPatch phase
vuln-pipeline patch <results_dir> runs after report (or instead of it). For
each unique crash: a patch agent in a fresh sandboxed container writes
a fix and emits a git diff; a separate grader container walks the
verification ladder — T0 apply + rebuild → T1 original PoC no longer crashes
→ T2 target test suite passes → re-attack: a 50-turn find-agent attacks the
patched target (for the rust profile, the patched crate re-driven through the
detectors, not a "binary"; the standalone vuln-pipeline reattack find→fuzz bridge
is the broader concept). T3 is an opt-in advisory style judge via --style; see
docs/patching.md. On a failing tier the evidence is fed back and the patch
agent iterates (≤5). Output: reports/bug_NN/{patch.diff, patch_result.json};
tier results land as t0_builds/t1_poc_stops/t2_tests_pass/re_attack_clean.
Before launching, check the target's config.yaml has a build_command.
Without it the grader can't recompile after applying the diff and the CLI
will error early. The shipped targets (7, across both profiles) have it.
Tell the user the ladder verifies the crash is gone, not that the diff is
safe to upstream. Surface patch.diff for human review and point at
docs/patching.md#reviewing-generated-patches for what to look for. Don't
offer to apply the diff to anything outside the pipeline containers.
For a quick patch demo without a prior find run, point at the cpp canary fixture:
vuln-pipeline patch targets/canary/fixtures/results_sample --model <m>.
For a rust end-to-end worked run, drive rust-canary through find → find→fuzz →
scorecard (the dvra3-parser DVRA benchmark takes the same path):
vuln-pipeline run rust-canary --auto-focus --aggregate union --model <m>, then
vuln-pipeline reattack results/rust-canary/<ts>/ --aggregate union --model <m>,
then vuln-pipeline scorecard results/rust-canary/<ts>/.
--accept-dos (off by default) lowers the find-agent's submission floor
for benchmark/validation runs. The default quality bar rules out DoS-class
crashes (allocation-size-too-big, stack exhaustion from unbounded
recursion) — agents triage them with allocator_may_return_null=1 and keep
hunting for memory corruption. Right behavior for real vuln hunting, but it
means DoS-class CVEs (e.g. wild-malloc via untrusted-size-field overflow)
land in transcripts without a formal submission. With --accept-dos, agents
submit the DoS find instead of skipping it. Use for known-CVE demo targets
where the CVE classification is DoS — check the CVE before launching.
--engagement-context <file> overrides the authorization block in the
agent system prompt. The pipeline ships a default "authorized security
research / defensive security assessment" block that's sufficient for most
open-source targets. If the user has an org-specific engagement scope
(authorized-by, in-scope subsystems, disclosure path), point this flag at a
file containing that block. Threaded to every agent's system prompt (find,
recon, report, grade, judge).
Duplicate handling
Two layers, both agent-judged:
Runtime (<dup_check>): find-agents must emit a <dup_check> tag
alongside <poc_path> with their reasoning for why the crash is distinct from
what's already in found_bugs.jsonl. The pipeline rejects submissions without
it. The agent makes the judgment (it knows root cause); the pipeline enforces
that the judgment happened. Entries in the jsonl are raw detector excerpts
(ASan SUMMARY / Miri diagnostic / panic backtrace + top frames) — agents
compare semantically, not by string match.
Report-gate (--stream only): a judge agent reads each graded crash
against the reports/manifest.jsonl and decides NEW / DUP_BETTER / DUP_SKIP.
Replaces regex signature-match as the gate — same root cause at different
lines/frames still dedupes. DUP_BETTER triggers a re-report; a compare agent
then picks the canonical one and writes canonical.json.
Writing known_bugs entries: key on function name, not line number. The
same bug crashes at adjacent lines or with different ASAN types depending on
input. See targets/README.md.
Architecture (what's load-bearing)
Two-container trust boundary. Find and grade run in separate containers built from the same image. Only the PoC bytes cross. The find-agent never touches the grade container — defeats reward-hacking via pre-positioned state.
The agent runs inside the sandbox, not on the host. bin/vp-sandboxed
sets the runtime/proxy env and execs the pipeline; each find/grade/report
agent then runs Codex -p inside its own gVisor container (--runtime=runsc
--network=vp-internal, egress = the configured allowlist via the proxy —
default api.anthropic.com:443; see docs/agent-sandbox.md for
Bedrock/Vertex). The agent's Read/Write/Bash are confined to that
container.
Agent-spawning subcommands refuse to start outside the sandbox unless
--dangerously-no-sandbox is passed.
Agent tool set is fixed: find/grade/report agents get Read, Write,
Bash. Judge, compare, and report-grader agents get no tools — everything in
the prompt, one short call each. No MCP, no web access.
Gotchas
Agent API requests carry a usage marker (anthropic-cyber-runbook header + a
cyber-runbook/ User-Agent token, 1P auth only). VULN_PIPELINE_NO_TELEMETRY=1
removes the pipeline marker; interactive sessions get a header-only marker
from .Codex/settings.json instead (override in .Codex/settings.local.json).
See docs/pipeline.md#usage-marker.
-e Codex= and -e IS_SANDBOX=1 in agent.py are load-bearing.
The first stops the in-container CLI's nested-session check; the second lets
it accept --permission-mode bypassPermissions. Don't remove either.
--tools not --allowedTools. The CLI accepts both but they're different:
--allowedTools is a permission allowlist; --tools actually restricts the
available set.
Canary parsers need __attribute__((noinline)). At -O1 they inline into
main, which changes the ASAN signature. If adding a canary parser, mark it.
Model is a runtime arg, not config. Always --model flag or
VULN_PIPELINE_MODEL env. TargetConfig has no model field by design.
Adding a target
Directory under targets/ with a Dockerfile (detector build for the target's
profile — rust: Miri/ASan/panic/hang; cpp: ASan) + config.yaml (profile: picks
the profile; rust targets also ship capabilities.json for detector routing). No
pipeline code changes. See targets/README.md.
Shipped targets (7): canary is the fast-iteration cpp smoke test (~6min, 3
planted bugs). drlibs, alsa, and htslib are real-world cpp CVE demo
targets — pinned to vulnerable commits, with per-target READMEs documenting
the CVEs and expected find times (htslib is the harder of the set: CRAM
container format, 10-CVE cluster). The rust targets: rust-canary (the rust
smoke test — seeded Miri-UB / panic / hang bugs plus a triage decoy), russcan
(the Vectorscan→Rust port's DB-parse surface), and dvra3-parser (the DVRA-3
benchmark — capability-routed, planted DVRA-003 stale-offset defect).
Tests
pytest tests/. Unit coverage spans tag/XML parsing, artifact serialization,
detector signature extraction, focus-area rendering, dedup signatures,
found_bugs.jsonl handling, the judge and compare agents, report parsing, the
T0–T3 patch-grade ladder, the /threat-model and /triage skill checkpoint
files, and system-prompt construction. The rust profile adds the find→fuzz bridge
(test_find_to_fuzz.py), union-of-N aggregation (test_aggregate.py), the
feedback loops (test_feedback.py), capability routing (test_capabilities.py),
and corpus replay (test_corpus.py). No integration tests — canary is the fast
cpp integration path and rust-canary the rust one
(--runs 3 --parallel --max-turns 50).
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

