codex-judge
Skill0 starsChanged 3 months agoMITView on GitHub
Gets an independent, cross-provider score on a code review verdict — a different model family than the one that implemented or reviewed the change, scoring correctness/security/maintainability/spec-compliance from the diff and the reviewer's verdict alone, never the implementer's reasoning, and gates auto-approve vs. auto-revise on the score. Degrades gracefully when no second provider is available. Use after a code review and before approving anything non-trivial, or when a review verdict feels uncertain or contested.
In this repository8 other files it gives its agents.
- advisorSkill
- compound-knowledgeSkill
- file-todosSkill
- goal-loopSkill
- metapromptSkill
- remote-agentsSkill
- skill-creatorSkill
- verify-before-doneSkill
The file
We split this skill into the rules it gives an agent
5 rules. About 0 are common, 3 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈3
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 25
After a code review, before marking a non-trivial change approved.
- Very specific: line 33
Confirm cross-provider access, or degrade honestly: Check whether a different-provider CLI is available and authenticated (e.g.
- Distinctive: line 47
Do not give it: the implementer's reasoning, self-assessment, or any narrative explaining *why* the change was made a certain way.
- Very specific: line 67
Ask the judge for an overall_score (aggregate 1–5) and a list of any critical violations (security holes, data loss risk, silently broken …
- Distinctive: line 80
If the judge and reviewer flatly conflict outside this matrix, prefer the reviewer when the disagreement is about code-context nuance the …
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Codex Judge
One model reviewing its own (or a same-family sibling's) work inherits the same blind spots that produced the bug in the first place — shared training data, shared architectural habits, shared things it systematically doesn't think to check. This skill gets a verdict from a genuinely different model family, structured to prevent that judge from just rubber-stamping the implementer's own framing.
This is the formalized, rubric-scored version of Rung 4 in
verify-before-done. Use
advisor instead for an ad hoc second opinion on a plan or open
question — this skill is specifically for gating a post-review completion decision.
When to use this
- After a code review, before marking a non-trivial change approved.
- A reviewer's verdict is uncertain, contested, or the change is high-stakes (auth, payments, data migrations, security-sensitive code).
- Skip it for trivial changes (typos, single-line fixes with no real judgment call) — the overhead isn't worth it.
Instructions
- Confirm cross-provider access, or degrade honestly. Check whether a
different-provider CLI is available and authenticated (e.g.
which codex,which gemini). If the implementer/reviewer used Claude, the judge must be a genuinely different model family — a second Claude instance is not cross-provider, no matter how "fresh" its context is. If nothing is available: skip the judge step, let the reviewer's verdict stand alone, and say explicitly "judge: unavailable" in your report. Don't silently drop this step. Build the judge's input — and withhold what would bias it. Give the judge:
- the task spec / acceptance criteria
- the diff itself
- the reviewer's structured verdict and findings
- automated check results (tests, lint, types) if available
Do not give it: the implementer's reasoning, self-assessment, or any narrative explaining why the change was made a certain way. That narrative is exactly what anchors a judge toward agreeing with the implementer instead of evaluating the artifact on its own terms. See
references/judge-prompt-template.mdfor a ready-to-use prompt built around this constraint.- Invoke it headlessly, with a fully self-contained prompt. The judge call has
no access to this conversation — everything it needs must be in the prompt (rubric,
diff, verdict, requested output format). Run it via whatever CLI is available
(e.g.
codex exec "$(cat judge-prompt.txt)") and capture the structured response. Score against this rubric — four dimensions, 1–5 each:
Dimension What it measures Correctness Does the change actually do what the spec asked, including edge cases? Security Any new vulnerabilities, unsafe patterns, or weakened guarantees? Maintainability Is the result clear, structured, and sustainable — not just working? Spec compliance Does it satisfy the acceptance criteria, not a narrower interpretation of them? Ask the judge for an
overall_score(aggregate 1–5) and a list of any critical violations (security holes, data loss risk, silently broken behavior) separate from the numeric score — a violation can sink an otherwise-high score.Apply the decision matrix:
Score Critical violation? Verdict Action ≥ 4 No PASS Auto-approve ≤ 2 — REJECT Auto-revise (spawn a fixer) any Yes REJECT Auto-revise, regardless of score = 3 No NEEDS_REVISION Reviewer's verdict is the tiebreaker If the judge and reviewer flatly conflict outside this matrix, prefer the reviewer when the disagreement is about code-context nuance the judge couldn't see (it only got the diff, not the live codebase); prefer the judge when it's flagging something the reviewer's own verdict didn't address at all.
- Report the verdict with the evidence, not just the label. State the per-dimension scores, any critical violations found, and which action follows — don't just say "judge approved it."
Reference files
references/judge-prompt-template.md— the self-contained prompt to send the second-provider CLI, including the requested structured output format.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

