agentleFS
guimatheus92guimatheus92/pr-review/.claude/skills/validate-recipe/SKILL.md

validate-recipe

Skill3 starsChanged 31 days agoMITView on GitHub

Verified build/test/run commands for pr-review. Used by /validate; also useful to any agent that needs to build, test, or launch this project.

  • Reads credentials
  • Deletes or force-pushes

The file

We split this skill into the rules it gives an agent

1 rule. About 0 are common, 0 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Never read-modify-write a UTF-8 doc through PowerShell 5.1 (Get-Content -Raw … | Set-Content): it decodes as ANSI and turns every →/— into …

    line 47

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Validation recipe — pr-review

Stack

TypeScript CLI (Node >= 20), esbuild single-file bundle at dist/cli.cjs; tests via node:test + tsx.

Tier 1 — static

  • npm run build # tsc typecheck + esbuild bundle; no separate lint script exists

Tier 2 — tests

  • Full suite: npm run test # scripts/test.mjs → node --test --import tsx, recursive over tests/**/*.test.ts
  • One file: node --test --import tsx tests/<file>.test.ts

Tier 3 — runtime

  • The surface is the bundled CLI: node dist/cli.cjs <command> (rebuild first).
  • Publication-policy offline fixtures: after building, node --test --import tsx tests/exit-code.test.ts drives the bundled CLI with a deterministic attempt-writing runtime, all five severities, full verifier/dedupe/artifact flow and verify --offline. It also drives standalone post through mocked HTTP for all three input shapes and proves input bytes stay unchanged. Verified locally: 9 tests pass. These are not real-provider publication evidence; the publication acceptance case in evals/acceptance/README.md still needs authorized provider credentials.
  • Offline full-pipeline dogfood (no PR, no network beyond packs): npm run build then npm run dogfood -- --base origin/main — converts the branch diff into a synthetic gather and drives the real bundle with --from-gather --dry-run. Add --context-only for routing only. Refuses a stale bundle, so build first. URL-based companions are off here by design.
  • Full pipeline with no LLM — a stub runtime. --copilot <path> overrides the binary and pins the copilot runtime, so a stub does the whole real run: node dist/cli.cjs review <url> --from-gather <gather.json> --dry-run --no-codex --no-companions --copilot <stub.cmd>. On win32 the stub is a .cmd doing node "%~dp0stub.mjs" %* — %~dp0 is right ONLY when the stub is launched by path, as --copilot <stub.cmd> does; a stub found through PATH (a claude.cmd for companion detection, see the 2026-10-15 addenda) must name its script by absolute path; the stub reads --add-dir from argv to find the run dir, parses dispatch-plan.json, and writes attempt-<n>.json under each reviewer.attemptsDir (plus reviewer.capabilityPath for installed-plugin passes). Batches arrive as separate invocations — count them in a file under the run dir, since the CLI numbers attempts itself. Withhold one reviewer's attempt file to force the automatic-recovery batch. The child's stderr is captured, not passed through, so assert on run artifacts rather than on the stub's own output. This is the only way to exercise dispatch, recovery and the audit paths offline; pair it with git show origin/main:dist/cli.cjs > <tmp>/cli-main.cjs for a same-scenario before/after.
  • Side-effect-free smokes: node dist/cli.cjs cache clear --pr <url> (proves URL parse end to end), a bad URL with --detach (must fail foreground, exit 2, no new dir under ~/.pr-review/runs), node dist/cli.cjs packs sync / packs list / doctor (packs clone + freshness + Linguist cache, no PR needed).
  • Offline pass-selection smoke: in a temp repo with .pr-review.yaml containing skill_packs: [], run node dist/cli.cjs review <url> --context-only — must exit 2 (zero passes on a code PR) while still rendering the ## Stack section.
  • Live smokes (gated on credentials — gh/az): --context-only against 1 real GitHub PR and 1 real ADO PR — must exit 0, ## Stack plausible for the diff, ## Passes non-empty with the expected glob hits.
  • File-list completeness smoke (gh only, read-only, ~30 API calls): from a directory that is NOT a checkout, node dist/cli.cjs gather https://github.com/OpenAPITools/openapi-generator/pull/24767 --no-cache must exit 1 with github listed 3000 of 9782 changed files — file list truncated … run git fetch origin master refs/pull/24767/head there and retry and leave nothing under ~/.pr-review/cache/github/OpenAPITools__openapi-generator/.
  • Completion-path live proof (not run here — needs a clone of a >3000-file PR): clone OpenAPITools/openapi-generator, git fetch origin master refs/pull/24767/head, then the same gather command from the clone — PASS when stderr says completed 6782 file(s) from git at <root> and the JSON holds 9782 entries. Until someone runs it, the completion path's live evidence is the temp-git-repo tests in tests/gather-cache.test.ts.
  • Cache-marker smoke: after a cached gather of a real PR, delete changedFilesComplete from the entry under ~/.pr-review/cache/<provider>/<scope>/<n>/, run gather <url> again — stderr must say cache entry predates the file-list completeness check — refetching, the entry is rewritten with the marker, and a third run is a cache hit.
  • pr-review-gather.json after a GitHub gather with --no-cache: metadata.changedFileCount equals changedFiles.length, changedFilesComplete is true, and there is no fullDiff key (retired in #26). Only on a fresh gather — a cache hit returns the stored payload verbatim, so an entry written by <= 0.11 still yields a fullDiff, which is expected and harmless.
  • Real (dispatching) reviews need provider auth AND a runtime on PATH — treat as not locally verifiable unless both exist. Run them from a COPY of the bundle (cp dist/cli.cjs $TEMP/cli-x.cjs) so a later rebuild cannot abort the run.
  • Config-dependent CLI smokes without touching the real ~/.pr-review: run the bundle with USERPROFILE=<tmp-home> HOME=<tmp-home> (Node's homedir() follows USERPROFILE on Windows) and a .pr-review/config.yaml written under it; env -u GITHUB_TOKEN -u GH_TOKEN … makes the auth outcome deterministic. Before/after against the pre-change bundle: git show origin/main:dist/cli.cjs > <tmp>/cli-main.cjs — the committed bundle is fresh by CI contract, so no worktree build is needed.

Docs / README changes

  • Anchors: gh api "repos/guimatheus92/pr-review/readme?ref=<branch>" -H "Accept: application/vnd.github.html" | grep -o 'id="user-content-[^"]*"' lists the ids GitHub actually generates; diff against grep -o '](#[^)]*)' README.md.
  • Badges: curl -s https://img.shields.io/<path> | grep -o '<title>[^<]*</title>' — the title is the rendered text (CI: passing, release: v0.10.0).
  • Render: Playwright MCP on https://github.com/<owner>/<repo>/blob/<branch>/README.md (add #<anchor> to land on a section). It blocks file: URLs — serve local previews with a one-line node http server. Screenshots may only be written under <repo>/.playwright-mcp/; move them to the scratchpad and rm -rf .playwright-mcp before committing. GitHub serves a cached blob for a minute after a push — load blob/<sha>/README.md to see the new commit.
  • Mermaid: GitHub strips HTML entities in labels; write #lt;/#gt; for angle brackets.

Gotchas

  • Never rebuild dist/cli.cjs while a detached review is in flight: the CLI hashes its own bundle into the authenticated run state and aborts finalization with CLI artifact changed (exit 21) — the reviewers' work is written but never accepted. Build first, then launch; or launch from a copy of the bundle.
  • tsconfig.json excludes tests/, so npm run build typechecks src/ ONLY and the suite runs under tsx (types stripped, never checked). Removing a method from an interface therefore flags nothing test-side: stale stub properties compile forever, and a claim like "nothing writes this field any more" has no compiler proof. Back that kind of change with a runtime assertion plus a tamper check, not with a green build (#26).
  • Run the full suite from Git Bash, not from PowerShell/node: tests/command-finder.test.ts spawns bash, and from PowerShell the PATH resolves it to C:\WINDOWS\system32\bash.exe (WSL), which fails with /bin/bash: line 1: cd: "": No such file or directory — on main too. For a countable result use the direct TAP form, since npm run test ignores extra args: timeout 480 node --test --test-reporter=tap --import tsx $(find tests -name '*.test.ts' | sort) > <file>, then grep ^# (tests|pass|fail) — redirect to a file, never pipe to grep: the RTK command hook swallows TAP on stdout and the pipe prints nothing.
  • Provider tests are pure-logic; nothing in tests/ hits the network. The git-completion tests in tests/gather-cache.test.ts create real temp git repos (need git on PATH, ~0.5 s each).
  • Pre-fix regression worktrees need node_modules — junction the repo's own, but use PowerShell New-Item -ItemType Junction -Path <wt>/node_modules -Target C:/.../pr-review/node_modules (forward slashes: a backslash before n gets expanded into a newline by shell heredocs). cmd /c mklink /j from the Bash tool silently no-ops here; the symptom is ERR_MODULE_NOT_FOUND: @octokit/rest. Remove with (Get-Item <wt>/node_modules).Delete() before git worktree remove, or the junction takes the real node_modules with it.
  • Tamper checks: edit the worktree copy with a node one-liner that fails when the target string is not unique; keep the replacement on ONE line (} else if (x) { → } if (x) {) — a replacement carrying a literal backslash-n (or a real newline, which is what a shell-expanded backslash-n becomes) compiles into a syntax error and a whole-file TAP failure that proves nothing — and the anchor must not contain \n either (it collapses and the script reports anchor not unique: 1); pick a plain-text anchor such as 5. For each issue found.
  • Claude Code's Bash tool on this Windows host (Git Bash + the RTK command hook): heredocs are not literal even with a quoted delimiter — a doubled backslash collapses to one (a \\d regex written through a JS string arrives as d), and a body containing shell-looking git … lines can be rewritten by the command hook and break the quoting. Write test/source blocks that contain either with the Write tool to the scratchpad, then cat >> them from Bash; build unavoidable backslashes with String.fromCharCode(92).
  • Never read-modify-write a UTF-8 doc through PowerShell 5.1 (Get-Content -Raw … | Set-Content): it decodes as ANSI and turns every →/— into mojibake across the whole file. Use a node script or the Edit tool; git diff origin/main...HEAD | grep -c 'â€\|â†' must print 0.
  • Exit codes: pipe the CLI to head and you capture head's status, not the CLI's. Run it bare with >/dev/null 2>&1; echo $? when the code is the evidence.
  • Posting behaviour is best proven with a small tsx script driving runPost against a stateful fake provider that records writes AND throws — a test that only asserts counts cannot show duplication. Copy it into a base worktree to get the before/after comment counts.

Addenda — 2026-09-09 (branch fix/resume-companion-accounting)

  • Read formatWarning's text without uninstalling anything. Run the bundle with USERPROFILE=<tmp-home> HOME=<tmp-home> against doctor — NOT plugins doctor, which probes copilot plugin list directly and never reads the runtime, so PR_REVIEW_RUNTIME there changes nothing. doctor resolves the runtime (PR_REVIEW_RUNTIME=copilot|claude) and renders each missing companion's remediation through installCommandFor. With no installed_plugins.json under the temp home both companions count as missing and the hint prints; git show origin/main:dist/cli.cjs > <tmp>/cli-main.cjs gives the before/after side by side. The same temp home makes review <url> --context-only render the > **Degraded:** block naming each uninstalled companion and the dispatch count it skipped — the only place that shortfall is visible, since nothing gates on it.
  • Prove a companion-accounting fix against a REAL artifact. ~/.pr-review/runs/<id>/companions.json and the raw-companion_*.json beside it are the pair that exposes a divergence: file mtimes say which attempt wrote each. pr-review verify <id> --offline reads those fields and prints the INV-DEL-01 row — it went from companion(s) with no output: … to N/N reviewer(s) delivered, 0 companion shortfall on PrecoPratico-Backend#715 and Frontend#1173. Note the row said PASS in both states: it grades "the run reported it loudly", not "delivery was complete".
  • Companions under copilot. The two runtimes keep SEPARATE plugin registries — installing under Claude Code leaves the Copilot side empty, and --runtime auto probes copilot first, so the default run dispatches none. Install with copilot plugin marketplace add anthropics/claude-code then copilot plugin install <id>@claude-code-plugins; the larger anthropics/claude-plugins-official is refused by Copilot's marketplace.json schema. A failed uninstall leaves an orphan entry in ~/.copilot/config.json whose cache_path no longer exists, and copilot plugin list keeps showing it.

Addenda — 2026-09-24 (branch fix/hidden-children, INV-HYG-04)

  • Console-window proof (Windows only). The window only appears when the parent has NO console — --context-only ignores --detach, so launch it yourself the way detach.ts does: a tiny node launcher doing spawn(node, [bundle, 'review', <url>, '--context-only', '--no-cache', '--no-codex'], { detached: true, windowsHide: true, stdio: ['ignore', log, log], cwd: <PR checkout> }) ×3, while a PowerShell loop polls Get-CimInstance Win32_Process every 200 ms. A window is an OpenConsole.exe whose parent is svchost.exe (Windows Terminal default-terminal handoff); stop them afterwards. Measured on PrecoPratico-Backend#749: installed 0.14.0 and origin/main 0.15.0 → 33 windows / 40 s for 3 runs; fixed bundle → 0 / 6 s. Extract the old bundle with git show origin/main:dist/cli.cjs > f from Git Bash — PowerShell 5.1 > re-encodes it.
  • Orphan-runtime proof without spending an LLM. Stub runtime: stub.cmd = @node "%~dp0stub.mjs" %*, stub.mjs = setInterval(() => {}, 1000);. review <url> --detach --dry-run --no-codex --no-companions --copilot <stub.cmd>, wait for the node process whose command line has stub.mjs … --add-dir … <run-id>, Stop-Process -Force the pid in run.pid — the cmd.exe in front of the stub dies with it, the stub survives — then status <id> (exit 21) or review <url> --resume <id> …: the stub is gone and progress.ndjson says killed its orphaned runtime session. The stub never answers copilot plugin list either, which is how the companion-probe leak surfaced: count leftover stub.mjs processes at the end.

Addenda — 2026-09-25 (branch fix/hidden-children, PR #44 review fixes)

  • Full suite on this change: 911 tests. On Windows, 909 pass and 2 skip; the SIGTERM test skips on win32 by design. On Linux (WSL, non-root), 907 pass, 3 skip (the win32-only tests), and 1 fails that fails identically at the PR head (see below).
  • The POSIX half without waiting for CI: WSL. wsl.exe here has Ubuntu with node. A Windows node_modules does not run there, because esbuild and tsx ship per-platform binaries. WSL's git cannot read a worktree whose .git points at C:/…, so pack the tree from Git Bash: git ls-files -co --exclude-standard > list; tar -cf src.tar -T list. Extract it under the Linux home and run npm ci --ignore-scripts. /tmp does not survive between wsl.exe calls, so write results under ~. The session runs as root, and there tests/exit-code.test.ts › "bundled publication fixture…" fails identically at the PR head and with the change (2/2 runs each) while ubuntu CI passes it. Compare against the head before reading a WSL-only failure as a regression.
  • Mutations on an uncommitted tree: mutate a copy, never the worktree. git checkout -- restores HEAD, not your last edit. Extract the same tar into the scratchpad and junction node_modules (see Gotchas). Then a script applies each mutation by a unique anchor, runs the test file that should catch it, restores the file and compares sha256. Read which test went red: a mutation that breaks more than its target is red for the wrong reason (one here made every run.pid unreadable and had to be redone). 22 Windows and 5 Linux mutations were all red at their intended test.
  • Orphan-runtime proof, re-run on the new bundle: 1 stub runtime found with --add-dir and --disable-builtin-mcps in its argv. The first status showed killed its orphaned runtime session (node pid N) under an unchanged ⏳ dispatch — … headline, and a second status kept that headline. 0 stub.mjs processes were left.
  • Closest to CI in WSL: a non-root user. Create one (useradd -m), copy the tree into its home and run through runuser -u <user> --. A previous root run leaves a root-owned /tmp/control.key that fails 9 single-session tests with EACCES, so remove it first. Clean non-root result at the PR head: 887 tests, 883 pass, 1 fail (the fixture above), 3 skip.
  • Make taskkill unreachable to prove the failed-kill paths. Set SystemRoot to a missing dir, import spawn.ts through its consumer (it captures System32 at load), then restore SystemRoot before spawning. Then drive the path with an idle .cmd stub and a short timeout: runCodexReviewer({ timeoutMs }), runSingleSession(opts, ctx, undefined, []) with opts.timeoutMs (it has no CLI flag, 30 min by default), and detectCompanions (30 s built in). At the PR head the first two stay pending; with this change they settle, printing could NOT kill … taskkill ENOENT. Kill the surviving stubs afterwards.
  • A fake binary must come first on PATH. On Node 24 here, a bare execFileSync('where', …) resolved from PATH before the cwd, so a fake where.exe only in the cwd was never run.
  • A CIM query launched from Git Bash matches itself. The filter text sits in the command line of its own powershell.exe and of the bash.exe wrappers, so -like '*stub.mjs*' counted 4 processes that were only the query. Filter by Name -eq 'node.exe', or run the query from the PowerShell tool.

Addenda — 2026-10-06 (branch feat/live-mod, the Claude Code mod)

  • The mod needs Claude Code >= 2.1.287 on PATH (claude --version); 2.1.263 validates the module (hooks:/calls: lines) but has no plugin test and never loads mods. npm run validate:mod prints the calls line, which is the read-only contract ($.fs.read/stat/exists/list, $.clock.*, $.env.get, $.ui.*, $.command.register); npm run test:mod runs the 35 kit tests in about eight seconds (one of them advances a mocked clock two hours). claude plugin test <dir> takes the MOD's folder and runs every *.test.ts beneath it — pointed at the repo root it loads the 65 node:test files and fails them all, which is why the mod lives under mods/live/ and is declared from .claude-plugin/plugin.json hooks; the Validating hooks: …mods\live\hooks\hooks.json line is the tripwire that the manifest path is honoured.
  • Mutation-check the kit tests like the node ones: copy register.js aside, break ONE anchor the way the Gotchas entry says — a node one-liner (or the .NET ReadAllText/Replace/WriteAllText from PowerShell, UTF-8 without BOM) that fails unless the anchor matches exactly once, because sed -i and .Replace() exit 0 on zero or several matches and a zero match leaves the test green for the wrong reason — then run claude plugin test mods/live, expect exactly the covering test red, restore. Anchors that worked: run.done || in tick, the silence note, if (run.preview), the text fallback. Two breaks were invisible until the tests changed: a torn first read is re-read regardless of any flag (the flag was redundant and was removed), and a timer that keeps firing after done only shows when the run finishes WHILE attached and the test counts ui.invalidate calls — a stub on ui.invalidate is honoured even though the kit answers it by default.
  • Live proof needs a terminal, not the VS Code chat panel. claude --plugin-dir <checkout> --debug-file mod.log: Read manifest hooks for plugin pr-review (enabled=true): ./mods/live/hooks/hooks.json names the file and hooks module pr-review@inline loaded (worker, …); events: session.start,session.end,tool.call,command.run,ui.render proves the load. A -p run with the same flag loads the module too; it registers nothing and reads nothing there (the headless guard), which the kit's two headless tests pin. The kit's launch test must use the command text the product sends — the Step 1 block of commands/pr-review.md verbatim, $CLI unexpanded — because a test that spelled an expanded cli.cjs path proved an attach the slash command never triggered (the "fixture answered its own question" trap; caught by the dogfood and the fresh review, not by 13 green tests).
  • Fixtures: node scripts/mod-fixtures.mjs <run-dir> mods/live/tests/fixtures/run-798.ts regenerates both snapshots from a real run; the running snapshot is the same run cut just after the 7th output-first-seen, with delivery-state.json rewritten to the in-flight shape single-session.ts writes during a batch. The reviewer's home is folded to C:\Users\dev and the file is grepped for the account name before it lands.
  • Throwaway worktree for tamper and before/after proofs, from a Claude Code worktree session. git worktree add <scratch>\wt HEAD then New-Item -ItemType Junction -Path <scratch>\wt\node_modules -Target <this checkout>/node_modules (PowerShell tool; forward slashes in the target) gives tsx to the copy. A single-file "pre-fix" is git checkout <base-sha> -- src/commands/verify.ts inside it, the base bundle is git checkout <base-sha> -- dist/cli.cjs there (no > redirection, so nothing is re-encoded), and git checkout HEAD -- <file> restores. Remove the junction FIRST ((Get-Item <wt>\node_modules).Delete()), then git worktree remove <wt> --force.
  • The Bash tool's worktree guard refuses commands whose text contains git, HOME=/USERPROFILE=, or sed with a variable operand (the RTK hook rewrites them, so the guard cannot read them) — even inside a printf string. Run git, env-scoped probes ($env:USERPROFILE = <tmp> before copilot plugin …) and ledger appends (Add-Content -Encoding utf8) from the PowerShell tool. Mutate a UTF-8 source for a tamper check with [System.IO.File]::ReadAllText / .Replace(…) / WriteAllText(path, text, UTF8Encoding($false)) — Get-Content -Raw | Set-Content would mojibake the mod's ✓ ● ◐ glyphs and fail the kit for the wrong reason. A PowerShell here-string commit message loses everything after a double quote; keep messages free of ".

Addenda — 2026-10-15 (branch fix/companions-duplicate-records, #54)

  • Companion detection under claude, offline, with any plugin list you want. A claude.cmd stub first on PATH answers plugin list --json; the CLI reaches it through spawnCli because --context-only never spawns the session. Two traps: the stub batch must name its script by ABSOLUTE path (invoked as "claude" through cmd, %~dp0 expands to the cwd, so @node "%~dp0stub.mjs" fails with Cannot find module <cwd>\stub.mjs and the CLI reports plugin list --json failed (exit 1)); and Git Bash resolves a bare claude to the npm one, so test the stub through cmd //c, never by calling it from bash. Point installPaths at real cache versions (~/.claude/plugins/cache/claude-plugins-official/code-review/<ver>) or at a temp dir holding .claude-plugin/plugin.json (name must match) plus the definition file. Read the verdict from companions.json (recognized, plannedDispatches, detectionWarning) in the run dir named on the **Run dir:** line — the single-session dispatch stderr line is printed AFTER the context-only return, so it never appears there.
  • doctor is detection only. It said ✓ companion code-review — 1 dispatch(es) on the bundle that then refused the plugin at materialization; review --context-only is the check that reaches materializeCompanionBriefs.

Last verified: 2026-10-15 against 9085f98 (PR #61)

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.