multi-review
Skill0 starsChanged 3 months agoMITView on GitHub
Use when reviewing almost any meaningful artifact, decision, action, plan, code change, prompt, skill, research summary, outbound message, or public-facing content. Runs a small panel of diverse review lenses across model families when available, synthesizes findings into fix/ask/defer/wontfix decisions, and iterates until the result is ready.
- Reads credentials
7 other projects use a version of this.
Adopted by
Most recently taken up by
- ngscheurich/skillstheir own version, first seen Oct 20, 2026
- steven112163/dotharnesstheir own version, first seen Oct 15, 2026
- Valeriy1980/claude-skillstheir own version, first seen Oct 13, 2026
- choms0521/my-claude-skillstheir own version, first seen Apr 22, 2026
- rozza/ai-skillstheir own version, first seen Apr 15, 2026
- ferologics/pi-skillstheir own version, first seen Feb 12, 2026
Other versions7 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 7
- PSPDFKit-labs/pi-skills★ 17same name
- ferologics/pi-skills★ 14same name
- choms0521/my-claude-skills★ 1same name
- ngscheurich/skills★ 1same name
- rozza/ai-skills★ 1same name
- steven112163/dotharness★ 0same name
- Valeriy1980/claude-skills★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository36 other files it gives its agents.
- check-upstream-firstSkill
- data-verificationSkill
- deep-diveSkill
- delegation-handoffSkill
- grok-searchSkill
- is-it-really-brokenSkill
- keep-goingSkill
- learn-from-conversationsSkill
- memory-cleanupSkill
- moa-solveSkill
- mob-checkSkill
- project-stewardSkill
- prove-the-setting-worksSkill
- recallSkill
- reportSkill
- robustify-doctorSkill
- skill-librarianSkill
- trust-frameworkSkill
- address-pr-commentsSkill
- diagram-renderingSkill
- find-the-real-bottleneckSkill
- live-database-maintenanceSkill
- mini-appSkill
- pr-review-sweepSkill
- scheduled-job-runnerSkill
- stop-the-noiseSkill
- upgrade-a-live-service-safelySkill
- vendor-switch-checkSkill
- writing-for-non-engineersSkill
- email-stewardSkill
- fieldySkill
- google-docsSkill
- google-sheetsSkill
- google-slidesSkill
- imessage-bluebubblesSkill
- vapi-callsSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
33 rules. 0 are common, 24 are distinctive, and 9 are very specific to this project.
0
Common
Most of the 5 edited versions we read keep these.
24
Distinctive
This author's own choices. Worth a look.
9
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 44
Use this skill when the user says:
- Distinctive: line 61
Do not use for:
- Distinctive: line 82
Run reviewers independently: Keep reviewer prompts isolated so they do not anchor on each other's conclusions.
- Distinctive: line 88
Meta-review: Check whether the review was useful, whether the panel missed an obvious lens, and whether false positives were handled …
- Very specific: line 93
Never claim "multi-model review" unless multiple model families actually ran.
- Distinctive: line 113
Use the strongest practical isolation mechanism available, but match it to the task's shape:
- Distinctive: line 125
Native subagents: when every seat can run on the _same_ model and you only need lens diversity: prompts, context, and failures are …
- Very specific: line 144
Same-model subagents with different lenses: when only one model family is available.
- Very specific: line 147
Manual single-pass review: only for quick/low-stakes work that is below every minimum depth floor (see below).
- Distinctive: line 170
If unsure, use balanced. Escalate to deep when any reviewer finds a high-severity issue or when the artifact will be hard to undo after …
- Distinctive: line 189
Prefer a reviewer from a different model family than the calling agent.
- Very specific: line 192
Use the models configured in the local Hermes profile.
- Distinctive: line 214
Read the scope honestly: that was one task, scored once, N=1 per model, and the task was _research and ideation_, not _critique of an …
- Distinctive: line 245
Verify which model actually answered before counting it as family coverage.
- Distinctive: line 299
Do not fight this by asking one seat to do both.
- Distinctive: line 335
Treat process commentary in a reviewer's response as a smell.
- Distinctive: line 383
Do not default to Grok for final wording, empathetic messaging, or careful policy synthesis.
- Very specific: line 403
Give each reviewer its own scratch home: Use the bundled helper scripts/reviewer_home.sh (the larger panel runner is …
- Distinctive: line 423
Reviewers are anonymous: Run 2 or 10; the helper never names or enumerates personas.
- Distinctive: line 525
If a run still times out, first try to recover the coverage — shrink the artifact, split the panel, or replace the seat — rather than …
- Distinctive: line 529
Never silently fall back to a partial review.
- Distinctive: line 577
When a reviewer says it could not read a seat's output, that its input was empty, or that a file was 0 bytes, verify the claim against the …
- Distinctive: line 600
Do not copy a version number out of this document.
- Distinctive: line 604
If the profile exposes a router combo/alias for a family, prefer the combo name over a raw slug.
- Very specific: line 609
If the profile routes everything through a custom multi-provider router, the providers will instead be custom aliases (for example …
- Distinctive: line 728
Do not ask every reviewer to solve everything.
- Distinctive: line 733
After reviewers finish, synthesize into action buckets.
- Distinctive: line 766
When a finding claims impact, separate four things that are easy to blur together:
- Very specific: line 828
If the user asked you to execute and the fixes are safe, clear what you're authorized to clear before reporting.
- Very specific: line 830
If the task is review-only, you don't edit the artifact — report each open Critical/High/Medium with a recommendation under edit/hold/block.
- Distinctive: line 836
Before declaring success, review the review:
- Distinctive: line 898
Calling it multi-model when it was not: If only one model family ran, stamp the run as degraded.
- Very specific: line 910
Ignoring missing setup: If the environment lacks Grok/Gemini/GPT routing, fall back honestly and note how to improve the panel next time.
“Common” is measured: kept by at least 6 in 10 of the 5 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Multi-Review
Overview
Multi-review is a generic review pattern: look at an artifact through several independent lenses, preferably using more than one model family, then synthesize the findings into concrete next actions.
This is not only an outbound-communication gate. Outbound comms are one important case, but the same pattern works for code, skills, plans, prompts, migrations, research, public posts, tool actions, config changes, incident reports, and decisions with real blast radius.
The point is diversity without chaos:
- Different lenses catch different failure modes.
- Different model families catch different blind spots.
- A synthesis pass deduplicates, rejects false positives, and turns feedback into action.
- A meta-review pass checks whether the review itself was useful or noisy before the caller declares the artifact ready.
When to Use
Use this skill when the user says:
- "review this", "sanity check this", "second pair of eyes", "is this ready?"
- "check this before I send/post/ship/merge/run it"
- "run a multi-review", "get Grok/Gemini/GPT on this", "use multiple reviewers"
- "look over this plan", "review this PR", "review this skill/prompt"
- "is this safe?", "what am I missing?", "red-team this"
Also use it proactively before high-stakes actions:
- Code or config changes that will be committed, deployed, or rolled across machines
- Public or customer-visible writing
- Messages sent as a human/operator
- Data migrations, deletes, permission changes, or credential-handling changes
- Skill, prompt, or agent-behavior changes that future agents will follow
- Research summaries where citations, uncertainty, or omitted evidence matter
Do not use for:
- Trivial private-chat responses
- Purely read-only information gathering
- Cases where the user explicitly says not to review or says "no further action"
- Emergency containment where pausing for review would increase harm; contain first, review the follow-up
Core Contract
A good multi-review run does these things, in order:
- Define the target. Identify the artifact or action being reviewed, the intended audience, and the stakes.
- Choose review depth. Pick quick, balanced, or deep based on risk.
- Choose a diverse panel. Select lenses and model families appropriate to the task. Whenever the run has two or more seats, staff for both generation and verification — those are usually different seats, not one strong seat (see "Verification and originality are usually different seats"). A single-seat run cannot split the roles; it carries the generator/verifier tension inside one prompt and is stamped degraded accordingly.
- Run reviewers independently. Keep reviewer prompts isolated so they do not anchor on each other's conclusions.
- Synthesize. Deduplicate findings, classify each as fix / ask / defer / wontfix, and decide whether the artifact is ready.
- Act. Apply obvious low-risk fixes when authorized; ask once for judgment calls; defer only when scope genuinely exceeds the task.
- Meta-review. Check whether the review was useful, whether the panel missed an obvious lens, and whether false positives were handled correctly.
- Iterate until ready. Re-run targeted reviewers after material fixes, especially for high-stakes or public-facing artifacts.
Never claim "multi-model review" unless multiple model families actually ran. If model
routing is unavailable, say degraded: single-model and explain what still ran.
Count a reviewer only when it returned substantive review output. A seat that merely
started, printed setup or shell noise, hung, or was killed is a failed seat, not an
independent lens. A same-family external reviewer does not by itself clear a
single-model-family degradation. Report completed seats and distinct model-family
coverage separately — they are different numbers and conflating them overstates the
review.
Do not stall silently. Give each reviewer a deadline. Once the expected window passes, poll for partial output rather than waiting indefinitely. If a seat times out or returns empty, replace it when the selected depth still requires that coverage; otherwise synthesize only if the completed seats still meet the depth floor for this target. A degraded panel that drops below its floor is not a finished review — say so instead of shipping it. Report which seats completed, which failed, and which families are missing, and lower confidence to match what actually survived.
Execution hierarchy
Use the strongest practical isolation mechanism available, but match it to the task's shape:
- Headless Hermes one-shots in isolated scratch homes
(
hermes -z... -t '', each with its ownHERMES_HOME) — the default. This is the only path that delivers a different prompt AND a different model per seat ("Grok, be critical"; "Claude, be empathetic"), and the only path where you can size the timeout to the job. Isolation is mandatory, not optional — see "Isolate every headless reviewer" below. Use a higher timeout than the default for real reviews; 300-600 seconds is usually reasonable, and deep/slow model panels may need the upper end. See execution rule 3 — the default is very likely lower than you want. - Native subagents when every seat can run on the same model and you only need lens diversity: prompts, context, and failures are naturally isolated. Two tradeoffs before choosing this path. First, the delegation tool has no per-task model parameter, and upstream has repeatedly declined to add one (PRs #17718, #23266, #25026, #34773, #36790; maintainer on #34773: "We do not want this"). Every child in a batch runs on the single configured delegation model, so this path cannot staff a multi-model panel — do not plan one around it. Second, a subagent's runtime almost certainly gives you no per-call timeout control: the schema exposes goal, context, role, and output schema — not a deadline. Any wall-clock cap is process-wide configuration read at call time, so a skill cannot scale it to the artifact. If this review needs a deadline proportional to its scope, use path 1 or 3.
- Parent-gathered I/O + reviewer one-shots for open-ended or I/O-heavy review work (large filesystem searches, email/search crawls, binary downloads, multi-step data collection). Do the I/O in the parent with normal tools, reduce it to a bounded brief, then send that brief to reviewers. Do not hand an open-ended crawl to a subagent whose deadline you do not control: if a wall-clock cap is configured, the child is killed mid-task and its findings die with it, and if none is configured a wedged child stalls the panel instead. Either way the review fails before synthesis.
- Same-model subagents with different lenses when only one model family is
available. Increase lens diversity, include at least one contrarian reviewer and one
meta-review, and stamp
degraded: model-diversity unavailable. - Manual single-pass review only for quick/low-stakes work that is below every
minimum depth floor (see below). Never use this mode for money/auth/secrets/user-
data/irreversible/public/rollout targets, even if they look small. Stamp
degraded: single-reviewerand do not present it as a panel.
Privacy applies to every path. The re-injection rule in execution rule 3 below is
not specific to --ignore-rules: any isolated reviewer — native subagent, headless
one-shot, or same-model subagent — runs without the calling context's project rules. If
the artifact may contain private data, or you are in a repo with a privacy/PII policy,
copy those constraints into every reviewer prompt regardless of execution path.
Depth Scaling
Quick — 1-2 reviewers. Use for low-stakes drafts, small edits, or a simple sanity check.
Balanced — 3 reviewers. Default for meaningful work. Cover the primary domain, truth/correctness, and user/audience impact.
Deep — 5+ reviewers plus meta-review. Use for security-sensitive changes, public posts, fleet/config rollouts, irreversible actions, architecture, migrations, or changes that future agents will rely on.
If unsure, use balanced. Escalate to deep when any reviewer finds a high-severity issue or when the artifact will be hard to undo after release.
Minimum depth floors
Certain targets should never get only a quick pass:
- Code that handles money, auth, permissions, secrets, user data, networking, or data migration → balanced minimum; deep if public or production-bound.
- Irreversible tool actions, deletes, permission changes, or fleet/config rollouts → deep minimum plus explicit approval/rollback review.
- Public-facing policy, docs, prompts, or skills that future agents will follow → balanced minimum; deep if the instructions affect safety boundaries.
- Messages sent as a human/operator, legal/medical/financial statements, or sensitive interpersonal comms → balanced minimum with empathy, evidence, and data-exposure lenses.
Model Family Selection
Prefer a reviewer from a different model family than the calling agent. Independence matters more than raw benchmark rank.
Use the models configured in the local Hermes profile. Do not hard-code API keys. If the
profile has aliases such as custom:grok, custom:gemini, or custom:openrouter, use
those; otherwise inspect the local config and choose equivalent provider/model pairs.
Pair each alias with the provider block the local config actually wires it to, and do
not "normalize" a reviewer onto a different provider for tidiness. When one router is
exposed through several provider blocks, they usually differ by API shape
(chat_completions vs anthropic_messages), and the better block depends on the
upstream model family: Claude models are natively Anthropic-shaped, most others (OpenAI,
xAI, Google) are natively OpenAI-shaped. The wrong block still returns HTTP 200, so this
fails silently rather than loudly. If an existing config contradicts that, assume it is
deliberate until you have checked why.
Family strengths
The notes below combine general observation with one measured data point: a multi-model bake-off where eight models were given the same open-ended research and ideation brief, and their outputs were scored across seven weighted dimensions (mandate discipline, evidence quality, originality, verification, usability, and related axes). Findings from that run are marked [measured]. Everything else is ordinary heuristic.
Read the scope honestly: that was one task, scored once, N=1 per model, and the task was research and ideation, not critique of an existing artifact. Behaviors that are themselves review-shaped — adversarial pressure-testing, mandate discipline, verification effort — transfer to reviewing with reasonable confidence. Generative traits like originality are a hypothesis about review behavior, not an established one. Treat the whole section as a prior to check against the artifact in front of you, not as fixed model properties.
- Claude / Anthropic — best for synthesis, nuanced tradeoffs, voice, empathy, policy interpretation, and turning messy findings into a coherent final answer. [measured] Also the strongest self-critic in the run: it pressure-tested its own output adversarially, which is review behavior directly. Its limit is divergence — it scored lowest of the frontier models on originality, and every idea it produced also appeared on another model's list. It is a convergence engine, not a divergence engine. A panel staffed only with Claude models will produce a well-written consensus and miss the objection nobody else thought of. Avoid using only Claude if the calling agent is already Claude-family.
- GPT / OpenAI — strong structured reviewer: code correctness, API contracts, tests, consistency, and concise fix recommendations. [measured] Best at building the evaluation scaffold, weakest at then executing it. It produced the best decision framework in the corpus — a crisp set of screening questions others should have been measured against — and then over-filtered its own candidates and delivered almost no verification behind them. Give GPT the job of defining the review criteria, or the job of applying them, but do not assume one seat does both well.
- Gemini / Google — strong long-context reader: large diffs, logs, docs, and cross-file consistency checks. [measured] Do not rely on it for evidence extraction, despite the long-context strength: its evidence was correct but generic and thin, list-shaped with no numbers behind the claims. Operational warning: in that run a Gemini seat silently fell back to a different underlying model mid-thread and kept answering as if nothing had changed. A reviewer that swaps model families without telling you breaks the independence guarantee the entire panel rests on. Verify which model actually answered before counting it as family coverage.
Grok / xAI — strong contrarian/red-team reviewer: assumptions, edge cases, blunt risk, adversarial misuse, policy gaps, and "what would embarrass us if true?" checks. [measured] The best mandate discipline in the run — it was the one model willing to answer the question actually asked rather than the more flattering adjacent question, and it rejected the framing it had been handed when the framing was wrong. That is precisely what a red-team seat is for. [measured] Its failure mode is delivery: it scored worst in the run on usability, producing genuinely excellent analysis and then leaving nearly all of it in scratch files while its actual reply led with process commentary. Grok needs an explicit output path and a demand for the artifact more than any other family (see rule 7). High variance is useful for surfacing issues, not for final wording.
Grok is also the family most likely to offer native live X-graph retrieval (via xAI's
x_searchserver-side tool) rather than general web search, which helps when the question is "how are people reacting right now?" — public-facing copy, launch posts, naming, positioning, and reputational blast radius. Live retrieval applies only when the selected route actually enables that tool; otherwise treat Grok as cutoff-bound like any other model.- Open-weight models (Kimi, Qwen, MiniMax, DeepSeek, Llama, and similar) — the divergence seats. [measured] In that run the open-weight models produced the genuinely novel material: the mechanisms and framings that appeared on no frontier model's list. If a panel needs an idea the consensus will not generate — an unlisted failure mode, an approach nobody considered, a structurally different objection — this is where it comes from, and it is a real reason to seat one even when a frontier model is available. [measured] They pair novelty with almost no verification: the same seats that generated the new material asserted it without checking, and one of them also scored near the bottom on usability. Their output is a lead to be verified, never a finding to be trusted as-is. Cost and privacy (local or self-hosted execution) are secondary reasons to use them; capability diversity is the primary one.
- Small or local models — useful for cheap/private quick passes, syntax/style checks, and obvious inconsistencies. Do not rely on them alone for high-stakes judgment. This is a size/deployment distinction, not the open-weight distinction above: a large open-weight model run through a hosted router is a full-strength seat.
Newer is not automatically stronger. A vendor's latest flagship, including one marketed as a reasoning improvement over the previous flagship, may not outperform the model it supersedes on your actual work. Two Anthropic frontier models in that run finished close together despite one being positioned as the clear successor. Check the newer model on a task you have already scored before promoting it to a panel seat by reputation.
These family strengths are observed heuristics, not guarantees — they shift with model versions and prompting. Verify against the artifact in front of you rather than treating them as fixed properties.
Verification and originality are usually different seats
The single most useful structural finding from that run: the models that verified heavily generated almost nothing new, and the models that generated the novel material verified almost none of it. The correlation ran in opposite directions across the whole field, frontier and open-weight alike.
Do not fight this by asking one seat to do both. Whenever the run has two or more seats, staff for it:
- Seat at least one generator (open-weight models are the measured pick) whose job is to produce candidate findings, including speculative ones.
- Seat at least one verifier (frontier models, Claude and GPT in that run) whose job is to check the generator's claims against the artifact and kill the unsupported ones.
- Let the verifiers grade the generators. An unverified novel finding is a lead, not a result, and it should be labeled that way in synthesis until someone checks it.
A panel of only verifiers returns a tidy consensus that misses the unlisted problem. A panel of only generators returns a pile of confident claims you cannot act on. The review is the interaction between them.
Single-seat runs are the documented exception. A quick-depth check or the
degraded: single-reviewer fallback has one seat and cannot split these roles. Do not
try to fake a panel out of it. Instead, make the tension explicit inside the one
prompt — ask for candidate findings and a verification pass over them, in that order
— and treat its novel-but-unchecked claims as leads, exactly as you would from a
generator seat. This is weaker than two seats, which is what the degradation stamp is
telling the reader.
Demand the artifact, not the summary
[measured] In that run the single largest score gap was not analysis quality but delivery: one model did roughly 94KB of excellent work, published about 1.2KB of it, and opened its reply with commentary about its own tooling — leaving a reasonable reader to conclude it had accomplished nothing. The work existed. It was simply never handed over.
This is the same failure class as rule 7 (incremental findings files), seen from the other end, and it changes what you ask for:
- Give every reviewer an explicit output path and require the findings to land there.
- Judge a seat by the file it produced, not by the chat message it returned. A thin reply over a substantial file is a delivery failure, not a weak review — go read the file.
- Treat process commentary in a reviewer's response as a smell. A seat narrating its helper scripts is usually a seat that has not yet told you what it found.
Retrieval is the weak link — paste the prior decisions in
[measured] In that run, none of the eight models consulted the existing record of what had already been tried, and several confidently re-proposed approaches that had already been evaluated and rejected, with the measured verdicts sitting in an accessible store the whole time.
Assume a reviewer will not find your prior decisions on its own, even when it has the tools and the access. A reviewer that re-raises a settled question burns a seat and adds noise to synthesis. Paste the relevant history — the graveyard of rejected approaches, the constraints already agreed, the decisions already made and why — directly into the reviewer prompt as part of the bounded brief. Retrieval you did not verify is retrieval that did not happen.
Job-to-family quick reference
Measured on one research task, N=1 per model. A starting prior, not a routing table — and never a substitute for the different-family independence rule above.
| Job | Start with |
| Synthesize findings into a coherent verdict | Claude |
| Write the evaluation framework or rubric | GPT |
| Apply a rubric and verify claims | Claude or GPT (not the one that wrote it) |
| Generate novel objections and unlisted failures | Open-weight (Kimi, Qwen, MiniMax,...) |
| Adversarial kill / mandate discipline | Grok |
| Read a large diff or corpus for coverage | Gemini |
| Extract evidence with numbers behind it | Not Gemini — Claude or GPT |
| Anything where the deliverable itself matters | Any seat — but give an explicit output path and check the file |
When Grok is the right pick
Grok earns a seat (or the lead reviewer slot) when the artifact needs adversarial or socially-grounded judgment rather than careful synthesis:
- Critical / red-team review — "tear this apart", pre-mortems, threat modeling, "what's the strongest argument this is wrong?"
- Public reaction and sentiment — how a post, product name, price change, or policy will land publicly; what critics will seize on. Treat this as platform reaction (X discourse skews fast, vocal, and manipulable), not a proxy for the general public.
- Current events grounding — claims that depend on what happened recently, where a stale training cutoff produces confidently wrong review notes.
- Contrarian check on consensus — when the other reviewers agree suspiciously fast and you want a dissent probe.
Do not default to Grok for final wording, empathetic messaging, or careful policy synthesis; its variance is a feature for finding problems and a liability for phrasing them. Pair it with a lower-variance synthesizer from a different configured family, and verify its claims rather than rubber-stamping them.
Isolate every headless reviewer in its own scratch home (REQUIRED)
hermes -z boots a full agent, and the CLI path opens the calling profile's
state.db read-write (cli.py:4642 → SessionDB(), cli.py:8566 →
create_session(...), resolved by hermes_state.py:2798). Run that from an
agent whose gateway is live and you have two OS processes writing one WAL
database — each with its own lock state and its own view of the WAL index. No
pragma prevents the damage.
This is not hypothetical. On a production host the gateway (fd mode u) and a
hermes -z reviewer (fd mode u) were caught holding one state.db
simultaneously. That 3GB database took structural B-tree damage — invalid page
number, 2nd reference to page, rowids out of order — and had to be rebuilt
offline from readable rows.
Give each reviewer its own scratch home. Use the bundled helper
scripts/reviewer_home.sh (the larger panel runner is
templates/parallel_reviewer_runner.sh):
source "$SKILL_DIR/scripts/reviewer_home.sh"
reviewer_pool_init # REQUIRED; never inside $( )
reviewer_run "$CRITICAL_PROMPT" -m grok &
reviewer_run "$EMPATHETIC_PROMPT" -m claude-think &
reviewer_run "$SECURITY_PROMPT" -m gpt-5.6-sol &
wait
reviewer_pool_destroy # or let the EXIT trap do itHERMES_HOME roots config.yaml, .env, auth.json, skills, memories and
state.db, so a seeded scratch home gives working credentials plus a private
database. Nothing is registered under profiles/, so there is no namespace to
garbage-collect and no name to collide with — the scratch dies with the run.
Reviewers are anonymous. Run 2 or 10; the helper never names or enumerates personas. The PROMPT decides what each seat is, so a panel can be whatever that day's artifact needs. Never hardcode a role vocabulary into the tooling.
One home per reviewer, never one shared home. Measured: 6 concurrent reviewers sharing a single home produced 6 simultaneous holders of one database — the original bug, relocated. Per-reviewer homes measure a peak of exactly 1. Integrity surviving one shared run proves nothing.
Measured cost: seeding a home ~1.2 ms / ~26 KB; state.db created on demand ~232 KB; 10 concurrent reviewers finished in 12.1 s using 2.7 MB of scratch, fully removed. Creating a home is far cheaper than the model call it wraps — never batch reviewers into one home to "save" it.
Five ways this helper can betray you (all measured, all guarded)
A reviewer panel found each of these in the first version of this helper. If you write your own, handle all five — every one fails silently.
- Unchecked
mktemp→ emptyHERMES_HOME→ the caller's live database. Hermes treats an emptyHERMES_HOMEas unset and falls back to~/.hermes/state.db. The isolation helper then causes exactly the corruption it exists to prevent. Validate every scratch path and refuse to run without one. - Lazy auto-init inside
$(reviewer_home)self-destructs. Command substitution runs in a subshell whose EXIT trap fires when the substitution closes, deleting the pool and handing the reviewer an unseeded home — the 401-with-rc=0 path.$$cannot detect a subshell (bash keeps the parent's pid) andBASHPIDis empty on bash 3.2 (macOS). Require explicit init. - A signal handler that does not exit lets the script resume. Bash returns control to the next statement, so a Ctrl-C'd fan-out destroys the pool and then seeds a fresh one and keeps spending model calls. Kill live reviewers, restore the default disposition, re-raise.
- A sourced
trap... EXITclobbers the caller's own cleanup. Chain it. - An exported pool variable is inherited by child shells, which skip init, adopt the parent's pool, and delete it on their own exit while the parent's reviewers are still running. Do not export it.
Two more that cost real seats: a trap cannot fire while bash blocks in a
foreground child (background each reviewer and wait), and auth.json must be
copied alongside config.yaml/.env or OAuth-based providers fail with
"No … OAuth credentials stored" while API-key providers succeed — a partial
credential failure that reads like a model outage.
Rejected alternatives — do not reach for these:
| Approach | Why it fails |
bare mktemp -d, unseeded | No credentials: HTTP 401: Missing Authentication header with exit code 0, so a fan-out silently scores dead reviewers as successful seats. |
a dedicated named profile + -p | Works, but litters the profile namespace with entries needing sweep-on-crash and forces invented names. Nested names fail rc=2 with empty output. |
| MoA presets | MoA broadcasts one prompt to N models. A panel needs N different prompts. Different feature. |
delegate_task per-task model | Upstream has declined it repeatedly (PRs #17718, #23266, #25026, #34773, #36790). It will not arrive — do not design around it. |
Running reviewers as Hermes one-shots
The cleanest way to run an independent reviewer is a headless hermes -z call against a
chosen provider/model. Confirm the local profile actually has the provider before using
it: hermes config get model.providers (or read ~/.hermes/config.yaml).
Keep reviewers on the configured router path — a slow reviewer is not a broken one.
A hermes -z reviewer call pays chat-session startup (config load, memory/Cortex
prefetch, skill scan, system-prompt build) on top of the model's own latency, so a
single reviewer can take a minute or more even when everything is working. That slowness
is not a model failure and is not a reason to drop to a same-model panel or to
"optimize" by POSTing a router/provider endpoint directly. The fix for slowness is a
generous timeout plus parallelism inside the configured Hermes/provider/router path
(including a custom OpenAI-compatible router) — never a bypass. The only exceptions are
the router being genuinely unreachable (for a diagnostic) or the human explicitly
approving a different architecture. For the concrete slow-vs-broken correction that
produced this rule, see references/slow-reviewer-timeouts-router-path.md. Six
execution rules that prevent silent failures:
- Mind the artifact size. A
hermes -z "$PROMPT"call places the whole prompt on the process argv, and command substitution likehermes -z "$(cat file)"does the same — it does not dodge the limit. Normal artifacts (a function, a small diff, a message) are fine. For large inputs — big PR diffs, full logs, multi-file dumps — argv can hitARG_MAX(Argument list too long). When the artifact is large, prefer chunking into per-file/per-section reviews or gather in the parent and send a bounded brief. Do not pretend a temp file plus$(cat...)solves this; it doesn't. - Always disable tools with
-t ''. A headless reviewer that tries to call a tool will hang waiting for an approval that never comes.-t ''keeps it a pure text-in / text-out review. Do not remove it when customizing. Set the timeout explicitly on every reviewer call. Never inherit the default. Review models take longer than normal chat, especially with long prompts or slow/deep models — but an agent terminal tool's default timeout is typically sized for ordinary shell commands (a few minutes at most) and will cut a healthy reviewer off long before it finishes. An omitted timeout is not "the value this skill recommends," it is whatever the environment happens to default to. Pass it on the tool call every time, scaled to scope:
Scope Timeout Single small artifact, one lens 300s Normal review, multi-file or multi-lens 300-600s Deep panel, slow/reasoning models, large brief 600s Anything you expect to exceed the foreground ceiling background + poll, no fg cap Foreground tool calls usually have a hard ceiling of their own (commonly around 600s), so a review genuinely bigger than that must run as a background process and be polled — not squeezed into a foreground call that will be killed. Check your runtime's actual foreground maximum rather than assuming 600s is available.
If a run still times out, first try to recover the coverage — shrink the artifact, split the panel, or replace the seat — rather than accepting the loss. Only when that fails does the degradation rule in the Core Contract apply: synthesize solely if the completed seats still meet this target's depth floor, and label the gap. Never silently fall back to a partial review.
- Use
--ignore-rulesdeliberately, and re-inject any safety rules you still need. It stops the calling profile's persona from washing out the review lens — but it also strips project rules. If the artifact may contain private data (real names, host paths, ports, secrets, internal context) or you're operating in a repo with a privacy/PII policy (for example anAGENTS.mdzero-PII block), copy those constraints into the reviewer prompt so the headless reviewer doesn't echo sensitive data into its output or any follow-up text. Independence of lens, not loss of safety. - Confirm the provider exists first with
hermes config get model.providers(or read~/.hermes/config.yaml) before selecting it. - Run a cross-family panel in parallel background processes — but never with shell
&. For a genuine multi-family panel, wall time should be the slowest single reviewer, not the sum, so run the reviewers concurrently. The Hermes terminal tool rejects foreground commands containing&backgrounding ("Use terminal(background=true)...") and also rejectsworkdirstrings containing shell metacharacters, so a one-shotcmd & cmd & waitpanel will be blocked. The working parallel pattern is: launch each reviewer withterminal(background=true, notify_on_complete=true)writing to a distinct/tmp/out_<lens>.txt, then read the files after all complete. Default for a cross-family (2+ model) panel: parallel background — the whole point of a panel is diversity, and parallelism is what makes waiting for slow-but-healthy reviewers affordable. Fall back to sequential foregroundhermes -zcalls only for a single-reviewer check or when background orchestration isn't available. Do not stampdegraded: single-modeljust because a reviewer is slow — only after a genuine failure or a blown timeout for the configured depth (300s for normal, 600s for deep). Early-degrade on slowness is the classic bug this rule exists to prevent. Confirmed in practice; parallel default reaffirmed after a later recurrence. Have every reviewer write findings to a file as it goes, not only at the end. A reviewer that is killed — by a timeout, a wedged tool call, a crashed child, a dropped connection — takes everything it found with it if its only output channel is the final return value. This is a real and expensive failure: a review can identify a genuine bug and then die before reporting it, leaving no trace that the bug was ever seen. Give each seat its own output file and have it append findings incrementally, highest-severity first, so a partial file is still useful evidence. Then, when a seat fails, read its partial file before declaring the seat lost. Findings recovered this way are real findings — carry them into synthesis, attributed to a seat marked incomplete, and let the depth floor decide whether the panel still stands. Never discard a dead reviewer's output unread.
This applies to every execution path, but it matters most where you do not control the deadline (path 1): a file on disk is the only thing that survives a child the runtime decides to kill.
- Never trust a reviewer's self-report about its own tool failures — stat the file. When a reviewer says it could not read a seat's output, that its input was empty, or that a file was 0 bytes, verify the claim against the filesystem before acting on it. Self-reported tool failures are biased: they fail in the direction that flatters the reporter, because "the input was missing" excuses an omission that "I did not read it" would not. This is not theoretical — in a real panel, a synthesizing model reported a peer's critique as "a 0-byte file" when the file on disk was over 7KB, and the dropped objection was the one that would have caught a regulatory risk on its own top-ranked recommendation. Check size and mtime yourself. A synthesis built on an unverified claim of missing input is not a synthesis, and the seat it silently dropped was usually the disagreeing one.
# Provider/model names are PLACEHOLDERS — resolve them from the local config.
# Suitable for normal-sized artifacts; for large inputs, chunk or send a bounded brief.
# When running these through an agent terminal tool, ALWAYS pass the timeout explicitly
# (see execution rule 3): ~300s normal, ~600s deep/slow panels, background+poll beyond
# the foreground ceiling. An omitted timeout inherits the environment default, which is
# usually sized for ordinary shell commands and will kill a healthy reviewer early.
hermes -z "$PROMPT_GROK" --provider <grok-provider> -m <current-grok-model> --ignore-rules -t ''
hermes -z "$PROMPT_GEMINI" --provider <gemini-provider> -m <current-gemini-model> --ignore-rules -t ''
hermes -z "$PROMPT_GPT" --provider <gpt-provider> -m <current-gpt-model> --ignore-rules -t ''Do not copy a version number out of this document. Resolve each family's current flagship from the local config or provider listing — hard-coded slugs go stale and silently 404.
If the profile exposes a router combo/alias for a family, prefer the combo name over a raw slug. A subscription-backed combo (for example an OAuth seat with a pay-per-token fallback) prefers subscription quota, but can still fall back to metered usage — so verify routing and cost policy rather than assuming a review is free.
If the profile routes everything through a custom multi-provider router, the providers
will instead be custom aliases (for example custom:grok, custom:gemini,
custom:openrouter) with router-qualified model IDs. Inspect the config and use
whatever families are actually wired up — the skill cares about family diversity, not
the exact alias.
Lens Selection by Scenario
Code or Pull Request
Default panel:
- Correctness / logic
- Security / data exposure
- Tests / regressions
- Error handling / reliability
- Architecture / maintainability for deep reviews
- Performance only when the code path is hot or data volume matters
Good model mix: GPT for structured code triage, Gemini for large diff/context reading, Grok for adversarial/security assumptions, Claude for synthesis.
Skill, Prompt, or Agent Behavior
Default panel:
- Trigger clarity: will future agents load it at the right time?
- Operational truth: are commands, paths, flags, and tool semantics real?
- Procedure quality: does it tell the agent what to do next, not just describe a concept?
- Safety/boundary handling: approvals, secrets, public/private data, irreversible actions
- Failure modes: what happens when a tool is unavailable, output is huge, or the model is wrong?
- Retrieval/readability: concise enough to load, structured enough to follow
Good model mix: Gemini for long-context/coverage, GPT for structure and command specificity, Grok for adversarial prompt misuse, Claude for final synthesis and voice.
Outbound Communication
Default panel:
- Empathy / recipient experience
- Intent fidelity: does it say what the operator meant?
- Evidence: are claims true and appropriately qualified?
- Data exposure: does it leak internal, private, or wrong-person information?
- Voice and audience fit
Good model mix: Claude for tone and empathy, GPT for concise edits, Gemini for evidence checking, Grok for blunt risk on sensitive messages.
Plan, Strategy, or Decision
Default panel:
- Assumptions and missing information
- Dependency/order-of-operations risk
- Blast radius and reversibility
- Concrete next actions and ownership
- Contrarian review: what would make this fail?
Good model mix: Grok for contrarian pressure, Gemini for context coverage, GPT for structured plan critique, Claude for decision synthesis.
Tool Action, Config Change, Migration, or Rollout
Default panel:
- Target correctness: right host/profile/file/channel/account?
- Blast radius and rollback path
- Approval gate: does this need human go-ahead?
- Verification: how will we know it worked?
- Data exposure / secrets
- Idempotency and partial-failure handling
Good model mix: GPT for step correctness, Grok for failure/edge cases, Gemini for large logs/configs, Claude for operator-facing summary.
Research, Writing, or Analysis
Default panel:
- Evidence and citations
- Omitted counterarguments
- Audience usefulness
- Overclaiming / uncertainty
- Narrative clarity
Good model mix: Gemini for source coverage, GPT for structure, Grok for skeptical counterarguments, Claude for prose synthesis.
Reviewer Prompt Pattern
Give each reviewer the same target but a different job. Keep prompts focused:
You are the <lens> reviewer in a multi-review panel.
[If the artifact may contain private data or you are in a repo with a privacy/PII
policy, paste those constraints here so this reviewer keeps them in force.]
Review target:
<artifact or action plan>
Context:
<audience, stakes, constraints, what done means>
Your job:
- Look only through the <lens> lens.
- Find issues that matter in practice, not theoretical nits.
- For each finding, include severity, confidence, evidence/location, why it matters, and
the smallest useful fix.
- Distinguish real issues from tradeoffs, false positives, and merely theoretical
improvements.
- Do not rewrite the whole artifact unless the lens requires it.
- Do not echo private data (names, host paths, ports, secrets) into your output.Do not ask every reviewer to solve everything. Reviewers are sharper when their scope is narrow.
Synthesis Workflow
After reviewers finish, synthesize into action buckets. Require evidence: each material finding should point to the artifact location, reviewer observation, or source fact that supports it. Unsupported claims are hypotheses, not findings.
Severity scale
- Critical — unsafe to proceed; likely security/data loss/financial harm, irreversible blast radius, or a direct contradiction of the task.
- High — material correctness, safety, trust, or operational risk; should be fixed or explicitly accepted before proceeding.
- Medium — real issue that can bite later or degrade quality; fix when low-cost, ask if tradeoffs exist.
- Low — polish, clarity, maintainability, or preference; do not block unless many lows combine into user-visible quality risk.
Deduplication rule
Merge findings when they share the same root cause, same practical fix, or same operational impact. Keep the highest severity/confidence from the merged set and note which reviewers caught it. Do not count the same bug three times just because three models saw it.
When reviewers disagree, reproduce — do not arbitrate
Two reviewers contradicting each other on a factual point is not a tie to be broken by seniority, model reputation, or vote count. Go back to the artifact and check. The cheapest decisive probe beats the most confident reviewer, and a reviewer asserting that a real citation is fabricated is itself a common review error — verify before deleting anything on a reviewer's say-so. Only genuine preference forks, where both positions survive contact with the evidence, go to the user as a choice.
Quantified claims need an evidence matrix
When a finding claims impact, separate four things that are easy to blur together:
| Question | What it establishes |
| Existence | the problem is real, with a located instance |
| Observed frequency | how often it actually occurred, measured not guessed |
| Measured impact | what it cost where it occurred |
| Addressable impact | what the proposed fix would actually recover |
Existence does not imply frequency, and measured impact does not imply the fix recovers it. Do not stack several estimates that all draw from the same underlying pool and then report the sum — that double-counts. If only existence is established, say so and label the rest as unquantified rather than attaching a number that will be quoted later as if it were measured.
Action buckets
Auto-fix — Apply immediately when authorized. High confidence, low risk, and no product/human judgment needed.
Examples: typos, broken commands, missing verification step, obvious factual mismatch, formatting, unused import, missing null check where a crash is certain.
Ask — Batch into one user question. Use when reasonable people might choose different fixes, behavior changes, public voice changes, security posture, or scope expansion.
Defer — Valid but outside current scope. Record clearly if follow-up tracking exists; do not use defer to dodge a flaw that blocks the current task.
Wontfix / false positive — The reviewer missed context or the fix is worse than the risk. Explain the reasoning briefly. Complexity is a real cost.
Verdicts and stop criteria
Two distinct ideas — keep them separate:
- Stop criteria decide when you may stop reviewing.
- Verdict reports the readiness of the artifact given what is still open.
A finding is actioned (you may stop reviewing it) when it is one of: fixed ·
asked · deferred-with-reason · wontfixed-with-reasoning ·
reported-with-recommendation. Stop criteria: every Critical, High, and Medium
finding must be actioned before you stop; Lows do not block. Don't keep reviewing for
theoretical improvements forever. Actioned ≠ resolved — a deferred Critical is actioned
but still open.
A finding is cleared only when it is fixed or wontfixed-with-reasoning. Anything
still requiring work or a human decision (asked, deferred, reported, held) is
open, regardless of severity.
The verdict is then mechanical:
- pass — no open Critical, High, or Medium findings; every one was cleared (fixed
or wontfixed). Only un-actioned Lows may remain. Ready for the next step. A deferred
or merely-reported Critical/High/Medium can never be
pass. - edit — open findings remain, but they are straightforward and applyable without a human judgment call.
- hold — an open finding needs a human decision before proceeding.
- block — unsafe, incorrect, or policy-violating to proceed in current form. Any
open Critical that was not cleared lands here (or
hold), neverpass.
If the user asked you to execute and the fixes are safe, clear what you're
authorized to clear before reporting; don't stop with fixable open Critical/High/Medium.
If the task is review-only, you don't edit the artifact — report each open
Critical/High/Medium with a recommendation under edit/hold/block. That satisfies
the stop criteria (actioned) without claiming the artifact is ready (it isn't pass).
Meta-Review
Before declaring success, review the review:
- Did the panel match the actual stakes, or did it over-focus on one domain?
- Did any reviewer hallucinate commands, requirements, or facts?
- Were false positives rejected with reasons rather than blindly fixed?
- Were all high-confidence findings actioned — fixed, asked, deferred-with-reason,
wontfixed-with-reasoning, or reported-with-recommendation (and Critical/High only
marked
passif actually cleared, not merely deferred)? - Is the final artifact simpler and safer, not just more complicated?
- Did the synthesis honestly label degraded conditions, such as unavailable models?
For deep reviews, run a final small prompt against one independent model:
Meta-review this review result. Did the panel miss a necessary lens, accept false
positives, overcomplicate the artifact, or fail to turn findings into action? Return only
material improvements.Iterate if the meta-review finds material gaps.
Output Format
Return a concise review report:
## Multi-review result
**Target:** <what was reviewed> **Depth:** quick | balanced | deep **Panel:**
<lens/model list; include degraded notes> **Verdict:** pass | edit | hold | block
**Auto-fixed**
- <issue> → <change made> (<reviewer/model>)
**Needs decision**
- <issue> → recommendation + tradeoff
**Deferred**
- <issue> → why out of scope / follow-up
**Wontfix / false positive**
- <issue> → why not applied
**Verification**
- <tests/checks/re-review performed>
**Meta-review**
- <what improved after reviewing the review, or "no material gaps found">Omit empty sections. If the verdict is edit and you are not authorized to edit,
include the proposed patch or rewrite. If the verdict is hold or block, explain the
smallest path to unblock.
Common Pitfalls
- Calling it multi-model when it was not. If only one model family ran, stamp the run as degraded.
- Letting reviewers see each other first. This creates anchoring. Run independent reviewers before synthesis.
- Over-fixing theoretical issues. Good review reduces risk; it should not turn clear work into defensive sludge.
- Skipping meta-review. The synthesis can be worse than the raw findings if it blindly accepts noise.
- Using the same panel for every task. Code, comms, plans, and rollouts fail in different ways.
- Forgetting the approval gate. A review can recommend an action, but irreversible changes, public sends, secrets, money, and broad rollouts still need human approval.
- Ignoring missing setup. If the environment lacks Grok/Gemini/GPT routing, fall back honestly and note how to improve the panel next time.
- Confusing optimization with defect discovery. The goal is meaningful risk reduction, not generating feedback for its own sake.
- Letting
--ignore-rulesstrip safety, not just persona. It also removes project privacy/PII rules. Re-inject any privacy or safety constraints the artifact needs into each reviewer prompt before ignoring rules. - Delegating open-ended I/O-heavy review work. Subagents are best for bounded reasoning. For crawls, searches, downloads, or broad file inspection, gather and reduce in the parent first, then send reviewers a self-contained brief.
- Committing private routing notes to a public skill. Profile-specific provider aliases, private endpoints, key env names, and owner-specific panel recipes belong in private profile references, not the public repo. Public guidance should describe the pattern and safety constraints, not private infrastructure.
- Treating a killed reviewer as a reviewer that found nothing. A seat cut off by a timeout has usually done real work. If it wrote nothing incrementally you have no way to know, which is why rule 7 exists; if it did, read the partial file before writing the seat off.
- Accepting a self-reported tool failure at face value. "The file was empty" is a claim, not an observation. Stat it. See rule 8.
- Staffing a multi-seat panel entirely with verifiers. An all-frontier panel converges: it produces a well-written consensus and misses the objection nobody listed. Seat a divergence source (open-weight models are the measured pick) and let the verifiers grade it. This applies to real panels; a single-seat quick check is a documented degraded path, not a violation of it.
- Treating an open-weight seat's novel finding as a result. Those seats generate the new material and verify almost none of it. Novel plus unverified is a lead; label it that way until a verifier checks it.
- Counting a seat as family coverage without checking which model answered. A route can silently fall back to a different underlying model mid-run and keep answering as if nothing changed, which quietly collapses your model diversity to one family while the report still claims several.
- Assuming a reviewer found your prior decisions. It almost certainly did not. Paste the rejected-approach graveyard and settled constraints into the brief, or expect a confident re-proposal of something you already killed.
- Promoting a model to a panel seat on reputation. A newer flagship marketed as a reasoning upgrade may not beat the model it supersedes on your work. Score it on a task you have already measured first.
Verification Checklist
- Target, audience, and stakes identified
- Depth chosen based on risk
- Lenses selected for the scenario, not from habit
- At least two model families used when available (or degradation stamped if not)
- Privacy/PII constraints re-injected into every isolated reviewer prompt
- Panel staffed for both generation and verification (2+ seats), or single-seat run stamped degraded with both roles carried in one prompt
- Prior decisions / rejected approaches pasted into the reviewer brief
- Confirmed which model actually answered each seat before claiming family coverage
- Every reviewer call passed an explicit timeout scaled to scope (never inherited)
- Each seat wrote findings to its own file incrementally
- Any failed seat's partial output was read before the seat was declared lost
- Reviewer claims of missing/empty input were verified against the filesystem
- Reviewers ran independently
- Findings synthesized into auto-fix / ask / defer / wontfix
- Safe auto-fixes applied when authorized
- Material fixes re-reviewed
- Meta-review completed
- Final report labels model/panel degradation honestly
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

