release-audit
Skill2 starsChanged 56 days agoMITView on GitHub
Pre-release audit of pdfnative-mcp — two parallel auditors (claims vs code; docs, counters and agent surfaces), an adversarial verifier, an agent-autonomy pass and a GO/NO-GO ledger under .audit/<version>/. Run by the maintainer before every release; never invoked by the model on its own.
- Commits and pushes
6 other projects use a version of this, 1 of them edited.
Adopted by
Most recently taken up by
- Nizoka/pkinative-mcpedited, first seen Nov 1, 2026
- f0909172434/rigorgraphtheir own version, first seen Oct 11, 2026
- mono/SkiaSharptheir own version, first seen Oct 8, 2026
- newton-physics/newtontheir own version, first seen Oct 8, 2026
- Akikukeo1/alignment-under-pressure-benchmarktheir own version, first seen Sep 1, 2026
- Omerfaruk-aydn/Agent_OStheir own version, first seen Jul 20, 2026
Other versions6 projects use a version of this.
Word for word · 0
None found.
Edited · 1
- Nizoka/pkinative-mcp★ 047% same
Wrote their own version · 5
- newton-physics/newton★ 6ksame name
- mono/SkiaSharp★ 5.6ksame name
- Akikukeo1/alignment-under-pressure-benchmark★ 1same name
- f0909172434/rigorgraph★ 1same name
- Omerfaruk-aydn/Agent_OS★ 1same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository4 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- .github/copilot-instructions.mdCopilot instructions
- llms.txtllms.txt
The file
We split this skill into the rules it gives an agent
2 rules. 0 are common, 1 is distinctive, and 1 is very specific to this project.
0
Common
Most of the 5 edited versions we read keep these.
1
Distinctive
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 13
Read ledger.md first for the ledger and verdict formats.
- Distinctive: line 51
Add a check for each of these to the auditor briefs.
“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Release audit
Audit the release described by $0 (default: the newest release-notes/v*.md) against everything that changed since $1 (default: the previous v* tag from git tag -l). The audit produces findings, never fixes: every fix goes through the normal edit → gate loop afterwards, and the ledger records what was fixed.
Read ledger.md first for the ledger and verdict formats. Each phase below hands a template to the agents it spawns; the agents return findings, you file them.
Ledger location
.audit/<version>/ — git-ignored (check .gitignore covers it before writing; it is NOT under test-output/, which Claude Code is denied to Read), so nothing here is ever committed. One Markdown file per report: auditor-a, auditor-b, verifier-1, auditor-d, verifier-2, then the ledger and the verdict (formats in ledger.md).
Phase A and B — two auditors, in parallel
Spawn both with the Agent tool in the same message (distinct angles; neither sees the other's report):
- Auditor A — claims vs code. Template:
auditor-a.md. Every claim in the release note and the top CHANGELOG entry is checked againstsrc/tools/, the shared fragments undersrc/,tests/,examples/andscripts/; at least one assertion per claim is reproduced with a command (a test, a script, atools/callagainst the BUILT server), not inferred from reading. - Auditor B — docs and agent surfaces. Template:
auditor-b.md. The README tool matrix,docs/AGENT_CONTRACT.md,llms.txt,docs/AI_GUIDE.md,docs/API_STABILITY.md§5, the livetools/listdescriptions,SERVER_INSTRUCTIONSand the prompts,server.json,docs/assets/ecosystem.jsoncounters, the sample baseline, the conformance corpus — every surface an agent or a human reads — compared with the behaviour that actually shipped.
Both write their report in the finding format of ledger.md (id, severity, claim, evidence command, observed, expected).
Phase C — adversarial verifier
Spawn one verifier (template: verifier.md) with both reports. It re-derives every finding from scratch — re-runs the evidence command, reads the cited lines — and stamps each one CONFIRMED | DOWNGRADED | REJECTED | DUPLICATE with a one-line justification. Auditors have about 10 % false findings; the verifier exists to keep them out of the ledger. A finding the verifier cannot reproduce is REJECTED, not "probably fine".
Phase D — agent-autonomy pass, then verify
Spawn Auditor D (template: auditor-b.md, section "Autonomy pass") with one question: can an agent that has only what an MCP host shows it — tools/list (names, descriptions, input schemas, _meta.examples), serverInfo.instructions, prompts/get, resources/list, plus llms.txt and docs/AGENT_CONTRACT.md — drive every 1.x feature without reading src/? It picks every feature the release note names plus a sample of older ones, writes the tools/call it would send from those surfaces alone, and sends it to the built server. Then a second verifier pass (template: verifier.md) over its findings.
Phase E — GO / NO-GO
Merge the confirmed findings into the ledger, then write the verdict file (both formats are in ledger.md):
- GO — no CONFIRMED finding of severity
blocker; everymajorhas a fix commit or an explicit maintainer waiver in the ledger. - NO-GO — otherwise. List the blockers first, each with its evidence command, so the fix loop starts from the ledger, not from memory.
Report the verdict, the counts per severity and per stamp, and the ledger path. Do not push, tag, open a PR or publish: scripts/release-prepare.ts and the maintainer take over from GO.
Driving the built server
The server speaks newline-delimited JSON-RPC on stdio and exits when stdin closes, so one call is one shell command — write the request lines to a file with Write, then node dist/cli.js < request.jsonl. Read the frames on stdout; the boot log is on stderr. tests/_stdio-session.ts is the same harness in test form, and scripts/helpers/server.ts calls callToolDirect() of dist/server.js without a transport.
Known blind spots
Add a check for each of these to the auditor briefs; they are where an audit of this server misses something.
- Two schemas per tool, kept in step by hand. Every tool has a JSON Schema (what
tools/listadvertises) and a Zod schema (what the handler enforces).tests/_fixtures/tool-shape.jsonproves the STRUCTURE of the first only; nothing proves the two agree on a bound, an enum or a nested key unless a test sends a value at the edge. For each new input, send one value the JSON Schema accepts and check the handler accepts it, and one it rejects. - Capability negotiation and the legacy fallback. The same binary serves MCP 2026-07-28 (
server/discover) and the 2025-erainitializehandshake, on stdio and on HTTP. A feature that only works in one era ships silently: the unit tests call handlers directly.tests/cli-stdio.test.tsandtests/http-modern.test.tsare the only places the eras are exercised. - stdout purity. One
console.loganywhere undersrc/— or in a dependency — corrupts the stdio framing for every host. The gate'ssmokeanddist-probesteps catch the built tree; an engine diagnostic that falls back toconsole.warn(a build path without theonDiagnosticsink) goes to stderr and is invisible to them. structuredContentversusoutputSchema. The read tools' output schemas are projectable (every property optional) so afields/verbosityprojection always validates — which also means a missing field never fails validation. Check new output fields on the default,summaryand projected paths, and on the error path.- Cache poisoning.
PDFNATIVE_MCP_CACHE_DIRserves earlier bytes for equal inputs. A new tool whose output depends on a secret, the clock, the network or an operator knob must be inNON_CACHEABLE_TOOLSor carry its dependency in the namespace (cacheNamespace()); a call carryingencryptis never cached. A miss here is silent and wrong. - The SSRF guard. The only egress is
src/network.ts, to operator-configured endpoints; a tool argument must never supply a URL. Grep new inputs forurl,uri,endpoint,href; a/URIwritten INTO a PDF is data, not egress. - The output sandbox.
outputMode: 'file'writes underPDFNATIVE_MCP_OUTPUT_DIRonly; the deprecated misspelt aliasPDFNATIVE_MPC_OUTPUT_DIRis still honoured. A new file-writing path that bypassessrc/output.tsis a blocker. TOOL_API_VERSION. A schema or error-code change that ships without the bump leaves hosts and the response cache on a stale contract. Difftests/_fixtures/tool-shape.jsonand the error table ofdocs/AGENT_CONTRACT.mdagainst the previous tag.- Byte identity is proven for the corpus only.
tests/_fixtures/samples.sha256.jsonholds the samples the generator produces; a behaviour change on an input no example exercises is invisible to it. The six PKI / TSA examples are not fingerprinted at all — vitest runs them. - Regex-driven
verify:docsrules are blind to wording: they prove counts, versions, links and presence, never that a sentence is true. - CI assumptions live outside the repo: Trusted Publishing needs npm ≥ 11.5.1 on the runner (publish.yml installs it) and the
npm-publishenvironment must exist and be bound on npmjs.com; a green local gate proves nothing about the publish job. - Auditors have ~10 % false findings — never file an unverified finding, and never let a
REJECTEDone reach the verdict. - Publish-related strings in a shell command trip the guard hook (
.claude/hooks/guard.mjs, wired to Bash and PowerShell): anything containingnpm publish,gh release,git pushorgit tag <name>inside a segment, a$( ), or an interpreter payload is refused. Write such strings into files with Edit or Write, never throughechoor a heredoc.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

