pr-review
AGENTS.md3 starsChanged 31 days agoMITView on GitHub
Generic, plugin-based PR review tool for GitHub, Azure DevOps, and GitLab, packaged as a plugin for Copilot CLI or Claude Code. Orchestrates parallel review passes — each one a skill applied by a generic agent — in a single agent session via the task tool (Copilot) / Task tool (Claude Code), retains every deduplicated finding locally, and posts every publication-eligible finding as a resolvable inline review thread (all findings by default). The bundle at dist/cli.cjs is the single-file distribution artifact.…
- Reads credentials
- Commits and pushes
Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.
Of the AGENTS.md files we read, the 33,315 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.
In this repository4 other files it gives its agents.
- CLAUDE.mdCLAUDE.md
- acceptanceSkill
- validate-recipeSkill
- helpSkill
The file
We split this AGENTS.md into the rules it gives an agent
1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 70
Every reconciliation test asserts assertNoDuplicateComments: no path may leave two comments at the same location with the same text.
≈ “Common” is an estimate: at least 1 in 5 of the 42,283 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
pr-review
Generic, plugin-based PR review tool for GitHub, Azure DevOps, and GitLab, packaged as a plugin for Copilot CLI or Claude Code. Orchestrates parallel review passes — each one a skill applied by a generic agent — in a single agent session via the task tool (Copilot) / Task tool (Claude Code), retains every deduplicated finding locally, and posts every publication-eligible finding as a resolvable inline review thread (all findings by default).
Build & test
npm run build # tsc + esbuild → dist/cli.cjs
npm run test # node scripts/test.mjs → node --test over tests/**/*.test.ts (990 tests, ~15s — on Windows run it from Git Bash: PowerShell resolves `bash` to WSL and fails `tests/command-finder.test.ts`)
npm run test:mod # claude plugin test mods/live → the live mod's 35 kit tests, offline, ~8 s (needs Claude Code >= 2.1.287 on PATH)
npm run validate:mod # claude plugin validate → the module's hooks:/calls: lines; the calls line is its read-only contract
npm run build:watch # tsc watch (re-run `npm run bundle` for esbuild)The bundle at dist/cli.cjs is the single-file distribution artifact. The slash command (commands/pr-review.md) finds it via $CLAUDE_PLUGIN_ROOT/dist/cli.cjs under Claude Code (falling back to ~/.copilot/installed-plugins/), locates the checkout whose git origin matches the PR URL (current dir, its subdirectories, then its siblings; a primary worktree beats a linked one), prints a project-skill count that approximates the loader's rule, and runs node "$CLI" review $ARGUMENTS.
Architecture
Two-layer model: slash command → Node CLI → single agent session (Copilot CLI or Claude Code) dispatching review passes as generic agents (subagent_type/agent_type general-purpose) via task() / Task().
src/cli.ts— commander entry, subcommand routingsrc/commands/review.ts— gather → selection → planned delivery → complete-only dedupe/post. Schema-v1--resumevalidates HMAC authority, PR identity, execution-mode transition, and hashes before targeted recovery (a live PR must still be open only to publish); legacy consolidated output keeps replay support, while legacy Phase 1 is diagnostic-only.src/commands/status.ts/src/commands/detach.ts— authenticated live/recoverable/terminal status plus detached execution. Exit 21 means targeted recovery is possible; exit 22 is terminal.src/dispatch/single-session.ts— materializes context/pass/companion files and an immutable plan, emits deterministic named dispatch-only task calls, adopts uniquely identified missing Copilot sidecars, promotes attempt outputs, performs one automatic selective recovery, assembles Phase 1/final output in Node, and runs a direct conditional verifier.src/dispatch/delivery.ts/src/dispatch/runtime-events.ts/src/util/atomic-json.ts/src/util/control-auth.ts/src/util/finalization-lease.ts— strictFinding[]delivery, bounded top-level Copilot JSONL parsing, missing-only create-once adoption, atomic create-only canonicals, authenticated started/completed/spawn-rejected attempt lifecycles, requested/resolved model and composed runtime-error evidence, attempt ceilings, digests, non-mutating control reads, Windows transaction-safe state, HMAC authority, and one recoverer/finalizer/poster per run.src/dispatch/runtime.ts— runtime selection (resolveRuntime, runtimeSpawnArgs, taskCall, normalizeModel);--runtime copilot|claude|auto(default auto: probes PATH, copilot first). Planned sessions add the materialized run dir while denying posting paths. Copilot removes shell aliases from the visible tool surface, requests non-streaming JSON output, and runs withCOPILOT_PLUGIN_DIR_ONLY=true(1is ignored by Copilot CLI 1.0.84-3); Claude keeps its explicit tool allowlist. MCP is denied at the PROCESS level, not only the tool level — denying themcp__*tools alone still lets every server boot (acmd.exe+conhost+npx+nodeeach on win32, every console window leaking) only to be unreachable.MCP_PROCESS_DENIAL: Record<Runtime, string>holds the per-runtime switch, so a new runtime fails to compile until it declares one. The two are NOT symmetric: claude's--strict-mcp-configis categorical (every config source ignored, and no--mcp-configis passed, so the run-dir.mcp.jsonis inventory/provenance only — no runtime loads it); copilot's--disable-builtin-mcpscovers built-ins and is completed one name at a time by--disable-mcp-server, so its reach is bounded bydiscoverMcpCapabilities.src/util/progress.ts/src/util/posted-marker.ts— the phase/heartbeat feed and posting idempotency guard. Schema-v1 decisions use authenticated authority;posted.markerin the run dir is diagnostic only.mods/live/hooks/register.js— the Claude Code mod (hooks module), declared by thehooksfield of.claude-plugin/plugin.jsonand deliberately NOT a roothooks/dir:claude plugin test <dir>takes the mod's folder and runs every*.test.tsbeneath it, so the repo root would drag the node:test suite in (observed: 60+the file did not load). Copilot's rootplugin.jsonnever names it, and Claude Code older than 2.1.287 ignores it. It is a READER of the run dir —progress.ndjson,reviewer-progress.ndjson(the per-reviewer timeline; the mod is its first reader in production),dispatch-plan.json,delivery-state.json,passes.json,error.txt— ticked once a second (the band redraws every tick) and re-read every second tick, each file only on a size/stamp change, neverstatus(which reaps). A run that died without a terminal phase (error.txt, newer than the last progress line) settles asfailedwith thepr-review status <id>hint; a file that exists but cannot be read is named on the band, never mistaken for a--context-onlypreview. It attaches only to a validated id (one path segment: no separator,.., leading./-or control character — an Azure DevOps id may hold a space) that exists under the runs root: the banner a--detachlaunch prints at the start of stdout (past thekey: valuelines the launch block prints first — the model composes that block fromcommands/pr-review.md, so their names, order and count vary: a live session addedcli: …, and a preamble pinned to two names missed the launch), a--resume <id>typed into the Bash tool, astatus <id>poll when nothing live is attached, or/pr-review-live— never adir:line or anything inside a foreground summary, because finding bodies are model output about PR content (a forgeddir:would have pointed$.fs.statat a UNC path). The slash command sendsnode "$CLI" review … --detachwith$CLIunexpanded, so the launch is keyed on the banner, not on the command text;mods/live/tests/live.test.tsruns that literal block, andtests/mod-surface.test.tspins it tocommands/pr-review.mdand the banner tosrc/cli.ts. The run dir is written by the CLI under the user's own account and is writable by the reviewer session itself, so the mod shows what it finds there and decides nothing from it —statusand the summary stay the authoritative readings. It draws the boxed band above the prompt (title and status, a bar with one stretch of cells per reviewer coloured by state, its legend, the counts, the sources — the PR label shortens below 96 and 64 columns, and under 7 rows of room the legend and the sources go), thepr-review-livepane (reviewers grouped under their source, one row each: state, time, short name, match kind) and the spinner suffix, registers/pr-review-live [run-id | off], strips control and Unicode format characters from every name and detail it shows, logs its own failures to the debug file only ($.ui.log … { to: 'debug' }), and stays inert whensession.startsaysisInteractive: false, because pr-review's ownclaude -previewer sessions load the user's plugins (only Copilot is isolated, INV-CTX-06). Its surface is pinned read-only bytests/mod-surface.test.ts, which also refuses a rewritten or mutated Bash event in the observer, any$alias, and a forbidden call on any receiver.src/dispatch/codex.ts— optional read-only Codex sibling; strict attempt-scopedFinding[]output and crash-safe attempt accounting (opt out:--no-codex)src/dispatch/line-snap.ts— snaps finding line numbers to the nearest valid diff line before postingsrc/dispatch/lockfile-digest.ts— the only code that knows thepackage-lock.jsonformat: reduces the two sides gather read (viaPrProvider.readFileAt) to aLockfileDigestand renders## Lockfile Digest; pure, never fetches (INV-FETCH-04's exception)src/dispatch/call-sites.ts— the## Call sitessection: names the declarations a diff touches and finds their references with one read-onlygit grepat the PR head (else local HEAD, named as such), bounded, never fetching (INV-CTX-07)src/providers/github.ts/azuredevops.ts/gitlab.ts— PR data fetchers + comment posters (GitHub inline comments go out as one review batch — ONE attempt,runPostowns retry because only it can reconcile — with per-comment fallback; GitLab posts per-discussion via plain fetch). Each one delivers a COMPLETE file list or says so: GitHub carries the PR'schanged_filesasPrMetadata.changedFileCount(itspulls/:n/filesstops at 3000 entries silently) and fetches no whole-PR text diff (pulls.getwith the diff media type 406s above 300 files, so it is unusable exactly on the PRs that would need it); Azure DevOps pages iteration changes to completion ($top=2000,nextSkip, a cursor that does not advance throws), drops folder entries before they can count toward the file guard, and synthesizes each patch from two whole-filegetItemcalls — which is why it, alone of the three, honours the content-narrowingoptsoffetchChangedFiles; GitLab mapschanges_count("N+"=changedFileListTruncated, because/diffsserves exactly the capped set)src/dispatch/parsers.ts— JSON / bracketed-markdown / section-header output parserssrc/dedupe.ts— Jaccard token similarity, strict/loose/off modessrc/config.ts— 5-level config merge (flags > env > repo yaml > global yaml > defaults)src/util/retry.ts— retry/backoff helper for transient posting errorssrc/packs/sync.ts/load.ts— skill packs: clone/pull configured git repos into~/.pr-review/packs/<name>/(fail-soft; >30 days withoutpacks syncwarns on every review) and load their skill files perinclude/excludeglobssrc/stack/linguist.ts/manifests.ts/detect.ts— stack detection: GitHub Linguistlanguages.yml(auto-downloaded to~/.pr-review/cache/, refreshed onpacks sync) tags changed files by language; dependency names parsed from manifests only when cwd's git origin is the PR's repo (identity viasrc/providers/identity.ts, so legacy*.visualstudio.comremotes match canonicaldev.azure.com); beyond the shallow root scan, changed manifests and manifests beside/above changed files are read, so a deeply nested project contributes its own dependencies; evidence stays categorized (canonical Linguist languages, ecosystems, full dependency names, dependency tokens, dependency groups) rather than collapsing into one tag bagsrc/dispatch/pass-select.ts— pass selection: project skills (repo/configured/forced) become shared CONTEXT in every pass (skills-project.md, never pass slots) — every match injects, no numeric cap, bodies inlined whole (no byte truncation); passes are ranked by EVIDENCE TIER — specific path glob > manifest-backed dependency/framework token > language-consistent type/manifest glob > exact stack tag (capMAX_STACK_PASSES=6) — plus EVERY baseline, which is contractual and rides on top of the total pass ceiling. A product guide cannot qualify from a bare language or a generic manifest alone, and product identity tokens must co-occur in ONE dependency group. Up to two installed-plugin passes may be admitted from repository/path/topic evidence. With no pack passes, project skills become the passes; overflow + unmatched + index-mode packs get materialized bodies referenced by digest-boundskills-index.mdshards.src/commands/packs.ts—packs list/packs sync/packs add/packs suggestsubcommandssrc/plugins/loader.ts— discovers skills from standard paths (.claude/skills,.claude/rules,.copilot/skills,.github/skills,.github/instructions,.agents/skills, plus the~/mirrors) and from configured dirs. Follows directory links (symlinks and NTFS junctions) ONE hop, in every discovery and configured dir: a link the PR added or changed (its path, or any parent of it, is in the diff) is refused BEFORE anything behind it is read and named as degraded coverage; a link met inside a linked directory is not followed. Under askillsroot a subdirectory is a skill only through<dir>/SKILL.md— a SKILL.md-less subdirectory is skipped (stderr warning when it holds.mdfiles), flat.mdfiles at the root are still skills, aREADME.mddirectory entry is never a skill;rules/andinstructions/roots recurse as before. Applies rule trust before same-name dedupe, and folds semantically identical mirrors silently while still warning on divergent same-name rules. It is the only walker for repo, personal and configured dirs (packs and installed plugins have their own):walkSkillDirs/loadFromDirinsrc/plugins/builtin.tswere removed because a second walker meant a second link policysrc/plugins/trust.ts— a rule file added or modified by the PR under review is UNTRUSTED input: it is dropped from both authoritative context and the on-demand index, and named as degraded coverage. Checks lexical AND real paths, so a changed later mirror cannot evict its unchanged counterpart; comparisons fold letter case and Unicode (NFC) on EVERY platform — win32-only folding let a PR committing.Agents/skillsbypass a macOS reviewer. Trust is by authorship, not location: content whose real path is outside the checkout is trusted when the PR did not author the link that reaches it AND the file is committed and clean in its home git repository (git ls-files+git status; aSKILL.mdalso needs its whole directory clean) — the same gate applies to every rule outside the checkout (linked, configured or personal), and a repository git cannot read is skipped, never trusted — on Windowsgit checkoutof a PR branch writes THROUGH an NTFS junction into the shared directory, so a planted file would otherwise become a trusted rule for every sibling repo's review. Untracked or modified files there are skipped and named; a directory under no git repository at all is trusted as the reviewer's local configuration (one stderr note per directory reached through a link). Nothing depends on one company's layout: any link, any discovery root, any OS, git or not. In-repo--skillfiles stay scoped and must live inside the checkout;--force-skillis the explicit overridesrc/plugins/installed.ts— host-agnostic discovery of installed plugins from their manifests, in BOTH runtimes: Copilot CLI (~/.copilot/installed-plugins/<marketplace>/<plugin>/) and Claude Code (~/.claude/plugins/cache/<marketplace>/<plugin>/<version>/, whose generic discovery path is read frominstalled_plugins.jsonbecause several versions can sit side by side). It validates active roots selected by the runtime (Copilot'sconfig.jsonis JSONC; Claude'splugin list --jsonsupplies the enabledinstallPath, scoped to the reviewed directory bygovernsCwdincompanions.ts, whose manifest identity must match — the version is recorded, never adjudicated). Skills are namespaced<plugin>/<skill>; declared paths and symlinks are constrained to the plugin root. Also builds the MCP capability inventory (capabilities.json) from trusted repo config, user config, and plugin manifestssrc/providers/identity.ts— canonical checkout/PR identity:canonicalPrAuthoritynormalizes legacy<org>.visualstudio.com, encoded HTTPS paths, andssh.dev.azure.com:v3/<org>/<project>/<repo>onto one authority, including the ADO project so a same-name repo in another project can't supply manifests or rulessrc/util/git.ts—gitTopLevel(); the pipeline resolves the checkout root once so a run from a subdirectory still finds repo config and rules.gitOut()/gitZ()run read-only git and THROW on failure (a failed read is never an empty result); rule provenance and the gather's truncated-list completion both go through themsrc/plugins/companions.ts— detects installed companion plugins (pr-review-toolkit, code-review); copilot viacopilot plugin list, claude viaclaude plugin list --json, which supplies each entry's enabled status, exactinstallPath, and scope. That list is not cwd-filtered — run from one checkout it still reports every project-scoped install on the machine — sogovernsCwddrops entries whoseprojectPathis not the directory under review (containment,foldPath-folded: pr-review may run from a subdirectory, and the same project appeared twice differing only by drive-letter case). Reading the list as already-scoped turned an unrelated project's installs intoconflicting active installationsand silently dropped all 7 dispatches — and so did two records of the SAME project on different versions (Claude Code writes one per spelling,C:\…andc:\…, and updates only one of them), for 36 reviews in a week (#54). There is no conflict gate any more: every enabled record governing the directory under review becomes a root, the version string is never compared, andagreedDefinitionis the integrity check — it compares the whole definition FILE (frontmatter and CRLF normalized, before any criteria extraction), so identical files across roots dispatch and divergent files drop THAT companion only, naming every root with a digest. A record whoseinstallPathhas no manifest naming the plugin is named in the Degraded block and ignored while a sibling root dispatches. Each runtime has its own registry — installing under one leaves the other empty, and--runtime autoprobes copilot first, so the machine that has them under Claude Code only still dispatches zero companions. Detection failure isunknown, never "not installed";companionReviewerNames()is the single source of up to 7 planned identities. Supplied active roots are constrained to each plugin, every root must provide agreeing definitions, and an unterminated header or shell/network/acquisition/fan-out/posting directive fails closed. Frontmatter is stripped lexically because its metadata is not execution authority — and so is a## When to invokesection, which addresses the HOST that dispatches the agent, not the agent: the officialcode-reviewer.mdsays "Spawn this agent on the freshly written files" there, and scanning it as criteria refused the companion over prose describing its own call sites. Removed, not exempted, so nothing inside it reaches the dispatched agent either. A companion that cannot be materialized is degraded coverage, never fatal —materializeCompanionBriefsreturns failures instead of throwing, andrunReviewdrops that id frominstalledCompanionsBEFORE the planned roster is fixed (planning it and not dispatching it is what INV-DEL-01 reads as missing output, which would turn the degradation back into an exit 2). One bad plugin.mdused to abort a run holding nine healthy passes and five working companions. Onecompanion-brief-*.mdper identity is hash-bound into the plan. Every companion runs asgeneral-purpose;code-reviewkeeps only its materialized-diff criteria, never its nativegh/posting workflow — and those criteria have TWO marketplace wordings (CODE_REVIEW_SHAPES): theclaude-code-pluginsHIGH SIGNALblock, and theclaude-plugins-official0–100 rubric with its 80 cutoff, which is what Claude Code's default marketplace and a teamenabledPluginslist install (all 14 cached official versions). Matching only the first refused every official install for a month;tests/fixtures/code-review-command/holds both real files byte for byte, and a body matching neither is refused naming both. The Degraded line states the cost (0 of 7 companion reviewers dispatched,N reviewer(s) lost).formatWarningtakes the runtime because the install command is not portable: slash commands under claude,copilot plugin …shell commands under copilot (it has no/plugin), from marketplaceanthropics/claude-code— the largeranthropics/claude-plugins-officialcarries the same plugins but Copilot'smarketplace.jsonschema rejects it
Key conventions
The guarantees themselves — what always holds, why, and what verifies it — live in
INVARIANTS.md, one addressable block per ID. This section carries the
implementation detail a contributor needs while editing these files, and cites the IDs
rather than restating them. Changing what the product guarantees means changing
INVARIANTS.md first, in its own commit. IDs are append-only: retire, never rename or reuse.
pr-review verify <run-id> grades a finished run against the whole list.
- No repo pollution. (INV-FETCH-03) All run artifacts go to
~/.pr-review/runs/<id>/. Never write files to the user's working directory — and that includes the checkout's.git: pr-review never runsgit fetchorgit checkoutand never writes a ref in the reviewer's repository (git credentials are not the API token, a fetch inside a detached child can take minutes, and it would import branch-authored objects). The truncated-list completion insrc/commands/gather.tsonly reads objects already present and otherwise fails with the exact fetch command for the user to run. - Clean output. (INV-POST-03) Posted comments contain only the finding body — no severity prefix, no bot chrome, no footer. (The
---separators live only in the localpr-review-summary.mdthatrenderSummarywrites; nothing separates posted comments, because each is its own thread.) - Full retention, eligible-only inline posting. (INV-POST-01, INV-POST-02, INV-POST-07) On a publish run every eligible finding lands as a resolvable review thread (GitHub review comments, ADO threads, GitLab discussions) — never a top-level issue comment. Lines outside the diff are snapped to the nearest valid diff line. On GitHub and GitLab, findings that can't anchor where they point (file outside the diff, or no location) are re-anchored to the first valid diff line with the original
file:linekept in the body (reanchorinsrc/commands/post.ts); on Azure DevOps threads are posted at the reportedfile:lineas-is (ADO threads are not limited to diff lines) and a finding with no location lands as a resolvable PR-level thread — a thread ADO rejects is reported as an error, never dropped.skippedexists only for--dry-run; below-threshold findings are suppressed, not skipped. Never reintroduce anissues.createCommentfallback. - Publication policy is Node-owned. (INV-POST-01, INV-POST-07, INV-OUT-01, INV-OUT-02)
--publish-min-severityis CLI-only onreviewandpost.src/util/severity.tsowns parsing, ordering and partitioning;--fail-onuses the same ordering but always evaluates allfinalFindings. Complete delivery and full dedupe precede durable artifact persistence and publication filtering. Never inject the threshold into prompts, change verifier gating, or hide suppressed findings from the summary. New dispatch plans are schema v2 and authenticateexecution.publishMinSeverity(NIT when omitted); v1 reads remain publish-all without rewriting their signed bytes. Resume rejects an explicitly conflicting threshold before recovery, including under--force-post. Retention is reconstructed withdedupeFindingsagainst the original gather, while eligible pending publication is separate. Authenticated markers accumulate exact confirmedfile:line:bodykeys; effective finalization mode records dry-run promotion without mutating the original plan.verifyrecomputes retention and eligibility instead of trusting artifact counts. Standalonepostnever rewrites its input or treats its metadata as policy authority. - A failed write is not proof that nothing was written. (INV-POST-04, INV-POST-05) POSTing a comment is not idempotent, so a 5xx/timeout means unknown: the server can commit and lose the response. The rules that follow from that, all in
src/commands/post.ts:- Providers make ONE attempt and throw —
postLineCommentandpostBatchComments, on all three providers.withRetryis for reads and other idempotent calls only. Retry lives inrunPost, because onlyrunPostcan reconcile first. - Read the PR back before deciding anything — before a retry, before the per-comment fallback, and before reporting a count (
readLanded/claim). - Unknown ≠ empty.
readLandedreturnsnullwhen the read fails, and anullmust never be treated as "nothing landed": the outage that 504s a write is the one that fails the read. Onnullthe run reports and stops rather than re-issuing — re-issuing is how 3 findings became 15 live comments in one reviewed scenario. - Identity is
file:line:body, not body. Two findings can legitimately carry the same body (dedupeWithinBatchonly folds same-file, ±3 lines), and a body-only match promotes a finding that was never posted — which then fillsposted.markerand locks--resumeout of recovering it. Comments with no file are never ours (inline-only invariant). - Reconciliation is one-way — errors may be promoted to posted, never the reverse. Demoting on a stale read would mark a live comment un-posted and make the next
--resumewrite it twice. - Every publish attempt writes authenticated posting state plus the
posted.markermirror, carryingverified. An unverified run fails closed on resume like a corrupt marker. Gating this onposted > 0is what left the incident run with no guard at all. --resumere-reads the PR before publication (on--dry-runtoo), unions confirmations rather than overwriting them, and adopts only comments matching an eligible finding it would post. It never removes retained findings or adopts a suppressed finding. A failed re-read aborts a publishing resume unless the operator explicitly uses the existing--force-postescape hatch.- Every reconciliation test asserts
assertNoDuplicateComments: no path may leave two comments at the same location with the same text. This is not theoretical: a 504-after-commit turned 56 findings into 112 comments while the run reportedposted 0 / errors 56.
- Providers make ONE attempt and throw —
- An incomplete file list is unknown, never empty. (INV-FETCH-01, INV-FETCH-02)
gather.changedFilesfeeds the rule-trust gate,.pr-review.yamltrust, the MCP-config gate and manifest reading, and a path missing from it reads as "unchanged" — i.e. trusted. So a provider list is never taken on faith:runGathercompares it with the provider's own count (PrMetadata.changedFileCount) and honourschangedFileListTruncated; on any mismatch it completes the list from the reviewer's checkout (git diff-tree -r -M -zfrom the single merge base, plumbing so reviewer diff config cannot reshape it, one hunks-only patch per missing file) whencwdMatchesPrsays that checkout is the PR's repository and both SHAs are present, and otherwise throws BEFORE anything is cached (exit 2,error.txtin detached mode) naming the counts and thegit fetchto run. A criss-cross history (two merge bases) or a shallow clone is refused too — git and the provider could diff against different ancestors. Cache entries carrychangedFilesComplete; an entry without it predates the gate and is refetched once. Providers paginatefetchChangedFilesto completion and set the count whenever the API offers one. This is not theoretical: Azure DevOps was reviewing every >100-file PR on its first 100 files ($topdefault) from 0.6 through 0.10. - The list of paths is complete; the CONTENT is only fetched where it can be read. (INV-FETCH-04) The 500-file guard (
MAX_FILES_GUARD, now insrc/dispatch/diff-filter.tsbesideapplyDiffExclusions—gather.tscannot importreview.ts) runs after gather, so everything fetched for a too-large PR is paid for and thrown away: onegetItemper added file and two per modified one, each a whole file body — measured live, 501 requests and 5.1 s on a 501-file ADO PR, then "PR is too large". Now 0.patchPolicyanswers "can this file's content still reach a pass?" and the two cost sites ask it —AzureDevOpsProvider.fetchChangedFilesbefore spending two whole-filegetItemcalls, andcompleteFromGitbefore spawning agit diff-tree -pper missing file. Paths are NEVER narrowed, only content; patch-less rows were already normal (binary, pure rename, mode-only, ADO deletions). The guard applies to everyrunGathercaller, sopr-review gatheron a >500-file PR returns paths without content — deliberate — whilepr-review postpassespatchesRequiredbecause it gathers ONLY to build the valid-line map, and an empty map is the 422-on-the-batch failure that call exists to prevent. Two things follow, and they are keyed on different conditions on purpose: a list assembled while withholding content is never cached (restored under wider excludes it would look like a whole diff), andGatherOutput.patchesOmitted— set only when nothing carries a patch — is read byearlyExitGatebefore the exclusions, because with every patch gone the byte clause sums to zero and waves the run through. The checkout's owndiff_excludesreach gather asrepoExcludesso a 600-file PR its config trims to 480 stays reviewable; they are branch-authored, so they may shrink the COUNT but never suppress a file — otherwisediff_excludes: ['**/*']in a PR would deliver a review of its own code with no diff. The one bounded exception: an excludedpackage-lock.jsonis read through the provider API at the two commits the provider's diff compares (two reads per lockfile, ten lockfiles per PR, never past the guard) and only its digest — never the content — lands inGatherOutput.lockfileDigests; a cache entry holding a lockfile but no digest is refetched once. - A synthesized patch is hunks, not the whole file. (INV-FETCH-04) Azure DevOps has no diff endpoint, so
synthesizePatchbuilds one from two whole file bodies;toHunksframes it with git's 3 lines of context. A hunkless whole-file patch is not wrong —validLinesFromPatchstarts its cursor at 0, so the numbers work out — which is exactly why it survived from the first release: it fails silently, by making the cache, the## Diffblock every pass reads, and the 2 MB budget scale with the size of the FILES. It also made every line "in the diff", so the ADO posting behaviour AGENTS.md documents was true only by accident.postingPolicystates it instead: ADO neither snaps nor re-anchors; GitHub and GitLab do both. UsepostingShape—runPostapplies it andresumeReviewandverifyboth RECOMPUTE it to recognize the run's own comments, so a fourth copy of the rule is a double-post (INV-POST-05) waiting to happen. - The CLI is the ONLY writer — dispatched agents never post. (INV-POST-06) Every dispatch prompt (review passes, materialized companions, verifier, the PR-description check, and the orchestrator itself) carries
NO_POSTING_DIRECTIVEfromsrc/dispatch/single-session.ts, and a session-context test fails if any dispatch line loses it. This is not theoretical: the officialcode-reviewcompanion's command allowsgh pr commentand instructs posting a top-level "### Code review" verdict — a live run posted one (Preco-Pratico/PrecoPratico-Backend#586) before the directive existed. When adding ANY new dispatch path, thread the directive through it. - Passes, not reviewers. (INV-CTX-04) Every review pass is ONE skill applied by a generic agent — no reviewer
.mdis ever dispatched. The only briefs this repo ships are the conditional verifier andinternal/pr-description(DESCRIPTION_BRIEF, planned on every dispatched review, removable only with--skip pr-description); neither is a skill or counts toward the pass cap. Pass names are<pack>/<skill>(repo skills keep their plain name). Knowledge comes from skill packs (git repos cloned to~/.pr-review/packs/<name>/, configured viaskill_packs— the one yaml key that REPLACES rather than pushes, so[]disables packs and a repo list overrides the global one entirely) plus the standard tool skill dirs (.claude/skills,.claude/rules,.copilot/skills,.github/skills,.github/instructions,.agents/skills) and any configured dir (--skills-dir,extra_skills_dirs,PR_REVIEW_SKILLS_DIR), selected the same way.src/dispatch/pass-select.tsseparates lenses from context: matched project skills inject into EVERY pass as authoritative rules, and the passes are pack stack hits ranked by evidence tier (specific glob > dependency > language-consistent weak glob > tag, cap 6) plus every baseline pointer, which rides on top of the total cap; everything else lands in digest-boundskills-index.mdshards for on-demand reading.applies_to/applyTo/pathsfrontmatter all route the same way;inject_intois deprecated (stderr warning, then ignored). - Untrusted input is anything the branch under review authored. (INV-TRUST-01, INV-TRUST-02, INV-TRUST-03) A rule file (
.claude/skills,.claude/rules,.github/instructions, …) added or modified by the PR — or reached through a link it added or changed — cannot instruct that PR's own review:src/plugins/trust.tsdrops it from BOTH authoritative context and the on-demand index, and the summary names it as degraded coverage. Same rule for.pr-review.yaml(config is re-loaded without the checkout's copy) and for repository MCP config.--skills-dir/extra_skills_dirs/PR_REVIEW_SKILLS_DIRare NOT a bypass: they are selected like repo skill dirs (targeted → scoped rule, untargeted → name+description relevance heuristic, unmatched → on-demand index; originconfigured, still applied when cwd is not the PR's repo) and trust-checked — a PR-changed file inside a configured dir is skipped even when the dir comes from an unchanged.pr-review.yaml. 0.10 injected them whole into every pass; observed live: 1.4 MB × 8 passes on a 5-file PR. The ONLY bypass is--force-skill <file|dir>— per run, CLI only. There is deliberately NO yaml or env key for forcing, so a committed.pr-review.yamlcan never pre-authorize branch-authored content. - A parseable review is not a completed review. (INV-DEL-01, INV-DEL-03, INV-OUT-01)
finalizeReviewcollectsoperationalFailuresand exits 2 for them even when findings parsed fine: a planned companion that delivered no output (or two), a missing/invalidraw-<reviewer>.jsonsidecar, a post that failed or could not be verified, a failed review prerequisite. Exit 0 means the pipeline completed — without--fail-onit says nothing about the finding count, and the CLI prints how many were retained. Never let a partial delivery render as a clean run. The mirror error is just as real: a resume re-dispatches unresolved reviewers, companions included, butwriteCompanionArtifactlives inrunReviewand never runs there — socompanions.jsonkept the interrupted attempt's verdict and a COMPLETE recovery exited 2 over seven companions whose outputs were in the same run dir.resumedCompanionFailuresnow reconciles the record against the outputs the resume holds, one-way (INV-POST-04's rule: a recorded failure may be cleared by evidence, never invented from its absence), and writes the cleared verdict back —pr-review verifygrades INV-DEL-01 from those same fields, so a stale artifact also fails the audit of a run that behaved correctly. - A runtime result can recover only its own missing attempt. (INV-DEL-04) Copilot JSONL is a fallback transport, not a second parser ladder.
runtime-events.tstrusts top-level events only, pairs deterministic task names throughtoolCallId, and accepts only exact validFinding[].adoptRuntimeReviewerAttemptis create-only and runs before the existing promotion path. Never replace an invalid attempt file or write a canonical sidecar directly from runtime output. - Single Phase-1 dispatch session, Node-owned completion. (INV-DEL-02) All passes dispatch in one runtime process via
task()/Task(). Node owns accounting, bounded selective recovery, aggregation, verifier gating, and the complete-only posting gate. The direct verifier and optional Codex sibling are separate sessions by design. - Stack-agnostic prompts. (INV-HYG-01) The prompt text this repo still ships (
PASS_RULES,VERIFIER_BRIEF,DESCRIPTION_BRIEF, the codex prompt) must never reference specific frameworks — stack knowledge enters a review only through skills. Enforced by a denylist grep intests/zero-passes.test.ts. - Stack, capability, runtime and model discovery are recorded. (INV-CTX-01, INV-CTX-02, INV-CTX-05, INV-CTX-06) Every run writes
stack.json,passes.jsonandcapabilities.json— including the resolved host runtime and requested model after normalization.autois a delegation: when Copilot emits one valid pre-dispatchsession.auto_mode_resolved, authenticateddelivery-state.jsonrecords that root route separately per attempt. It is not per-reviewer served-model evidence. Selective reviewer redispatch reuses the initial route; a schema-v1 Copilot Auto run without it cannot complete or recover rather than inventing an alias. Use--default-modelto request a specific model. Finding nothing is a valid result; not recording it is not. Empty stack evidence must carry anotesaying why. - No window, no orphan. (INV-HYG-04) Every
node:child_processcall insrc/passeswindowsHide: true, and none passesdetached(it voidswindowsHideon win32) except the--detachlauncher. A detached run has no console, so on Windows a console child started without it gets a fresh console that Windows 11 hands to Windows Terminal as an empty window (measured: ~10 per review, one per synchronousgit/ghcall, each blocked ~3 s by the handoff; eight concurrent reviews froze the desktop).tests/windows-hide.test.tsis the whole guard — INV-HYG-04 istests-onlyinverify— so it is strict: the call-site count is pinned (a new site changes it on purpose), every import shape ofchild_processother than a namedimport { … } from 'node:child_process'is refused, a spread options object counts only as what its declaration in the same file says,.kill(on any receiver butprocessis refused, and everyspawnClichild must be registered withkillOnExit. Each rule has a control that proves it can fail. Children are killed with everything they started (killTreeinsrc/util/spawn.ts):taskkill /T /Fon win32, where they run behindcmd.exe; on POSIX SIGKILL to the process and each descendant the process table lists (an unreadable table kills the root only and is reported as not complete). A kill is reported only once the pid is gone (a zombie is gone); a failure always names image, pid and reason — on timeout on stderr plus anerrorline inprogress.ndjson, at exit on fd 2.killOnExitcoversexitand SIGINT/SIGTERM/SIGHUP, which Node's default disposition would end without it. A hard kill runs no handler, sostatus(a deadrun.pid, a dispatch plan, and no summary newer thanrun.pid) and--resume(before it claimsrun.pid, refusing while a session survives) runreapAndRecord: only inside the runs root, it kills processes whose argv carries the run dir with a flag only this CLI's argv has (--add-dir <dir>plus the MCP-process denial, or Codex's-C <dir>plus--skip-git-repo-check), re-reading each target andrun.pidjust before the kill, and records killed / survived /scanErrorunder thereapprogress phase. By argv, not a recorded pid — measured live, thecmd.exein front of the runtime died with the CLI and left the runtime unreachable through it.taskkillandpowershellrun from their absolute System32 path (a bare name is searched in the cwd first, and the cwd is often the PR's checkout), and the process table is read as UTF-8. - esbuild bundle. (INV-HYG-02)
dist/cli.cjsis a single-file zero-dependency bundle. Nonpm installneeded at the plugin install site.
Testing
Tests use node:test + node:assert. Run with npm run test. Tests are in tests/ mirroring src/ structure. Every test is hermetic: provider tests inject a stub Octokit client, GitApi, or fetch, so none needs a credential or the network. That is also the limit of the suite — nothing in it proves a real Azure DevOps or GitLab call works, which is what npm run acceptance exists for.
The live mod's tests are the one exception to the layout: they live in mods/live/tests/ and run with npm run test:mod (claude plugin test mods/live — offline, no sign-in, but it needs the Claude Code CLI, which CI installs on ubuntu). claude-code/testing imports do not load under node:test, and a kit test placed under tests/ would fail the node suite. The kit has no file access, so the fixtures are a TypeScript module, mods/live/tests/fixtures/run-798.ts: a pruned copy of a real run dir in two snapshots, running and done, regenerated with node scripts/mod-fixtures.mjs <run-dir> <out-file>.
A green test is not proof the behaviour is right — it can be proof the defect was written down twice. Three ways this suite has been green over a live bug, none of them findable by reading the test, all of them found by running the product:
- The assertion transcribed the current output.
tests/runtime.test.tsassertednormalizeModel('copilot', DEFAULT_MODEL) === DEFAULT_MODEL— that the shipped default reaches the CLI verbatim. It did, and Copilot CLI answersModel "claude-opus-4.8" … is not availableand exits 1 before dispatching anything, so every reviewer read as failed to deliver. When the assertion is about a value crossing a boundary you do not own, assert what the far side accepts, never what this side happens to send today. - The fixture made the comparison impossible to fail. An early INV-TRUST-01 check compared
passes.json'ssource(absolute) against the changed-file list (repo-relative), so no PR-authored skill could ever match and the check PASSed on every input, including the ones it exists to catch. Its test supplied a relative source — a shape the product never writes. A check that compares values from two different producers has to be fed the real shape of both, copied from an actual run dir. - The fixture answered its own question. See evals/acceptance/README.md — the control handler in the acceptance fixture carried a doc comment that contradicted its own code, the reviewer flagged it (correctly), and the false-positive assertion failed. A control has to be genuinely correct, not merely intended to be.
The common shape: each one was written by looking at the code instead of at what the code talks to. npm run test cannot tell you a real CLI rejects your model string, a real run dir writes absolute paths, or a real reviewer is right — npm run acceptance and npm run dogfood can.
Run artifacts under ~/.pr-review/runs/<id>/ are part of the contract, not debug spill: passes.json (routing), stack.json (detected stack), companions.json (installed / recognized / planned / completed dispatches), hash-bound companion-brief-*.md, capabilities.json + capability-<pass>.json (MCP inventory and per-pass usage evidence — both runtimes deny MCP at the process level, so the brief instructs passes to record all three arrays empty. A sidecar that reports otherwise raises a named degraded warning listing the fields and servers it claimed — the run still completes, because Node has no view of the session tool surface and so cannot tell a real denial leak from a fabricated one. The brief deliberately keeps that escape hatch so a denial that stopped working can still surface here, and pr-review verify raises the same claim as an INV-CTX-05 FAIL — an audit read after the fact is exactly where an unexplained claim should stop the reader, even when the run itself does not fail on it), attempt state with requested/resolved model and structured-result adoption evidence, raw-<reviewer>.json (one canonical sidecar per pass, companion and the description check), reviewer-progress.ndjson (the per-reviewer timeline — attempt starts, first-seen outputs, promotions, invalidations, recoveries, verifier steps — which the live mod reads and INV-OUT-02 lists), error.txt on any failure, and posted.marker on any publish attempt. --resume, the operational-failure checks and the live mod read them, so a change that stops writing one is a behaviour change.
Common tasks
- Audit a finished run against the guarantees:
pr-review verify [run-id](or--pr <url>). Read-only; one PASS/FAIL/SKIP row per invariant, always the full list, exit 2 on any FAIL.--offlineskips the live PR read,--jsonfor CI. - Prove it works on every provider and runtime:
npm run acceptance— 3 providers × 2 runtimes against real PRs, posting for real. See evals/acceptance/README.md; the estate is seeded once bynpm run acceptance:seed. - Add/curate a skill pack: Edit
DEFAULT_PACKSinsrc/config.ts(entry: git source +include/excludeglobs, optionalref/mode/baseline), or per-user viaskill_packs:in yaml (remember: it replaces the whole list) /pr-review packs add <owner/repo|url>(materializes the defaults first, then appends and clones). - Sync packs:
pr-review packs sync— clones/pulls every configured pack and refreshes the Linguist cache. Needed once per machine before the eval harness; >30 days without it warns on every review. - Run the eval harness:
node scripts/eval.mjs [case]— needsnpm run build, a real runtime on PATH, and synced packs; fixtures inevals/fixtures/(diff.patch + expected.yaml, optionallypr_title/pr_descriptionin expected.yaml, alockfile-digests.jsonthat becomesGatherOutput.lockfileDigests, and arepo/that is committed with the diff applied as the PR head in a temp git repo the CLI runs inside, so## Call siteshas callers to find). Assertions live inscripts/eval-assertions.mjs:must_find,must_not_find,distinct_findings(one generic finding can't satisfy two expectations),must_dispatch/must_not_dispatch, andstack/dependenciesinclude/exclude. A case may be negative-only — that is how a defective/control pair proves the finding is causal rather than vacuous. - Dogfood the local branch (no PR needed):
npm run dogfood -- --base origin/main [--include-untracked]— turns the branch/working-tree diff into a synthetic gather under~/.pr-review/runs/and drives the real bundled CLI with--from-gather --dry-run. Refuses a stale bundle (npm run buildfirst) and refuses secret-bearing names/content before writing artifacts. Companion coverage is intentionally off for a synthetic gather; exercise it on a real-PR dry-run. - Add a provider: Implement
PrProviderfromsrc/providers/types.ts, wire insrc/providers/index.ts, and teachsrc/providers/identity.tsits remote forms (checkout identity + cache scoping depend on it). - Change config defaults: Edit
src/config.tsDEFAULTSobject. - Change auto-discovery paths: Edit
autodiscoveryPaths()insrc/config.ts(repo rule dirs and their~/mirrors). - Force a skill past its declared scope:
--force-skill <file|dir>— a file, or every.mdunder a directory, injected whole into every pass with no scope and no trust check.--skill <file>KEEPS the file'sapplies_to/applyTo/pathsscope, so the bypass has to be asked for — and asked for on the CLI: there is no yaml/env equivalent on purpose. - Change dedupe behavior: Edit
src/dedupe.ts. Threshold constants are at the top. - Change diff exclusions, the file guard or the patch budget: Edit
src/dispatch/diff-filter.ts—DEFAULT_EXCLUDES,MAX_FILES_GUARD,MAX_PATCH_BYTES, andpatchPolicy, which decides what is worth fetching, all live there together. - Change runtime spawn args or model mapping: Edit
src/dispatch/runtime.ts. - Change Codex reviewer behavior: Edit
src/dispatch/codex.ts. - Preview pass selection:
pr-review review <url> --context-only— writes pr-context.md + per-pass files, prints the## Stackblock (languages, dependencies, notes) and the## Passestable (Pass | Matched by | Matched on | Source) + index count, exits without spawning the runtime (exit 2 when zero passes on a code PR). - Resume a killed run: schema-v1 runs validate authenticated state and redispatch only incomplete coverage; complete runs replay directly, and a complete dry run may be resumed without
--dry-runto post what it previewed (an incomplete one may not). Legacy Phase 1 cannot publish. Authenticated posting state prevents duplicate repeat resumes (--force-postoverrides posting only, never delivery trust gates). - Run in the background / check a run:
pr-review review <url> --detachreturns a run-id immediately;pr-review status <run-id>shows the live progress feed, or the summary once done. This is how the slash command avoids the host's ~10-min Bash timeout. - Check the environment:
pr-review doctor— runtimes on PATH, resolved runtime/model, codex + companions, provider auth, skill packs (git on PATH, per-pack state/freshness, Linguist cache), config sources. - Change the live mod: edit
mods/live/hooks/register.js, keepnpm run validate:modlisting only read calls ($.fs.read/stat/exists/list,$.clock.*,$.env.get,$.ui.*,$.command.register), and runnpm run test:mod. Develop against a session started in a terminal withclaude --plugin-dir <checkout> --debug-file <log>: the module hot-reloads on save, and the log'shooks module pr-review@inline loaded … events: …line proves it loaded from the manifest path. Write each$call in full and eachon('event')as a literal — the validator reads them from the source — and never callstatus, write a file or start a process (tests/mod-surface.test.tsrefuses it). The VS Code chat panel draws no mod: prove a change in a terminal. - Cut a release:
node scripts/release.mjs <patch|minor|major|x.y.z>— bumps every manifest, verifies no stale version string, rolls CHANGELOG, rebuilds, commits and tags. It stops there: pushing (git push --follow-tags) and creating the GitHub Release are manual, and a pushed tag alone never shows on the/releasespage — see CONTRIBUTING.md. - Before opening a PR: review the diff, don't just validate it. Build/tests/smokes prove the change works; only a review of the diff catches a doc that promises more than the code delivers, a constant deleted out from under its comment, or a behavior validated on one corpus and shipped product-wide. Dogfood it: push the branch, open the PR, run
pr-review review <pr-url> --dry-runagainst it, and triage before asking for human review (PR #16 and #17 both shipped real defects that only this step caught). When any guarantee wording changes (docs/CHANGELOG), grep every mode the code path has — a guarantee is universal unless the doc scopes it. - Editing files from a script: replace anchors, never slice ranges, and never through a shell heredoc. Two ways this repo has lost working code to a "safe" scripted edit. A patch script that rebuilt a file from the text between two markers silently dropped
credentialEnvfromscripts/acceptance-reset.mjs— everything between the markers that the script did not know about went with it; a one-cell smoke run is the only reason it was caught. And a Git Bash heredoc collapses backslashes, so"\\n"in the payload reaches disk as"\n"— it brokeexpected.yamland a fixture twice before the pattern was believed. Write the file with the editor tool, ors.replace(anchor, () => replacement)in a Node script where the replacement is a function (a string replacement re-interprets$&); assert every anchor matched before writing, and re-read what landed.
Maintaining review quality
The review knowledge lives in skill packs, not in this repo. Keeping quality up over time:
- Packs own the knowledge. The defaults (
DEFAULT_PACKSinsrc/config.ts) point at upstream repos;pr-review packs syncpulls them, and a pack unsynced for >30 days warns on every review. Baseline pointers (each pack'sbaseline:skill list) break loudly, not silently: a pointer that no longer resolves surfaces as amissingBaselinewarning — fix the pointer when upstream renames or removes a file, don't let it rot. - A real production miss becomes an eval fixture — as a causal pair. Distill the miss into
evals/fixtures/<case>/(diff.patch + expected.yaml) rather than expanding prompt text, and add a<case>-safecontrol whose diff is the corrected version and whose expectations are the same regexes undermust_not_find. The control moves exactly ONE variable: when the defect lives outside the diff — a false description (pr_description), a lockfile the diff excludes (lockfile-digests.json), a caller that never reads a new field (repo/) — the diff stays byte-identical and that input moves instead (tests/gather-from-patch.test.tspins this for the shipped pairs). A positive-only fixture passes on a fixture so broken that anything would be flagged; the pair is what proves the finding tracks the defect. Usedistinct_findings: truewhen two separate issues must each be found. Then runnode scripts/eval.mjs <case>(needsnpm run build, a real runtime, and synced packs). - The remaining prompt text stays stack-agnostic.
PASS_RULES,VERIFIER_BRIEF,DESCRIPTION_BRIEF, and the codex prompt must never name a framework —tests/zero-passes.test.tsgreps a denylist over them, so a hardcoded framework name fails the suite. Stack-specific rules belong in a pack or a repo skill.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

