agentleFS
jsvillalbatjsvillalbat/sevito-skills/skills/codex-judge/SKILL.md

codex-judge

Skill0 starsChanged 3 months agoMITView on GitHub

Gets an independent, cross-provider score on a code review verdict — a different model family than the one that implemented or reviewed the change, scoring correctness/security/maintainability/spec-compliance from the diff and the reviewer's verdict alone, never the implementer's reasoning, and gates auto-approve vs. auto-revise on the score. Degrades gracefully when no second provider is available. Use after a code review and before approving anything non-trivial, or when a review verdict feels uncertain or contested.

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 3 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    After a code review, before marking a non-trivial change approved.

    line 25
  2. Very specific:

    Confirm cross-provider access, or degrade honestly: Check whether a different-provider CLI is available and authenticated (e.g.

    line 33
  3. Distinctive:

    Do not give it: the implementer's reasoning, self-assessment, or any narrative explaining *why* the change was made a certain way.

    line 47
  4. Very specific:

    Ask the judge for an overall_score (aggregate 1–5) and a list of any critical violations (security holes, data loss risk, silently broken …

    line 67
  5. Distinctive:

    If the judge and reviewer flatly conflict outside this matrix, prefer the reviewer when the disagreement is about code-context nuance the …

    line 80

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Codex Judge

One model reviewing its own (or a same-family sibling's) work inherits the same blind spots that produced the bug in the first place — shared training data, shared architectural habits, shared things it systematically doesn't think to check. This skill gets a verdict from a genuinely different model family, structured to prevent that judge from just rubber-stamping the implementer's own framing.

This is the formalized, rubric-scored version of Rung 4 in verify-before-done. Use advisor instead for an ad hoc second opinion on a plan or open question — this skill is specifically for gating a post-review completion decision.

When to use this

  • After a code review, before marking a non-trivial change approved.
  • A reviewer's verdict is uncertain, contested, or the change is high-stakes (auth, payments, data migrations, security-sensitive code).
  • Skip it for trivial changes (typos, single-line fixes with no real judgment call) — the overhead isn't worth it.

Instructions

  1. Confirm cross-provider access, or degrade honestly. Check whether a different-provider CLI is available and authenticated (e.g. which codex, which gemini). If the implementer/reviewer used Claude, the judge must be a genuinely different model family — a second Claude instance is not cross-provider, no matter how "fresh" its context is. If nothing is available: skip the judge step, let the reviewer's verdict stand alone, and say explicitly "judge: unavailable" in your report. Don't silently drop this step.
  2. Build the judge's input — and withhold what would bias it. Give the judge:

    • the task spec / acceptance criteria
    • the diff itself
    • the reviewer's structured verdict and findings
    • automated check results (tests, lint, types) if available

    Do not give it: the implementer's reasoning, self-assessment, or any narrative explaining why the change was made a certain way. That narrative is exactly what anchors a judge toward agreeing with the implementer instead of evaluating the artifact on its own terms. See references/judge-prompt-template.md for a ready-to-use prompt built around this constraint.

  3. Invoke it headlessly, with a fully self-contained prompt. The judge call has no access to this conversation — everything it needs must be in the prompt (rubric, diff, verdict, requested output format). Run it via whatever CLI is available (e.g. codex exec "$(cat judge-prompt.txt)") and capture the structured response.
  4. Score against this rubric — four dimensions, 1–5 each:

    DimensionWhat it measures
    CorrectnessDoes the change actually do what the spec asked, including edge cases?
    SecurityAny new vulnerabilities, unsafe patterns, or weakened guarantees?
    MaintainabilityIs the result clear, structured, and sustainable — not just working?
    Spec complianceDoes it satisfy the acceptance criteria, not a narrower interpretation of them?

    Ask the judge for an overall_score (aggregate 1–5) and a list of any critical violations (security holes, data loss risk, silently broken behavior) separate from the numeric score — a violation can sink an otherwise-high score.

  5. Apply the decision matrix:

    ScoreCritical violation?VerdictAction
    ≥ 4NoPASSAuto-approve
    ≤ 2—REJECTAuto-revise (spawn a fixer)
    anyYesREJECTAuto-revise, regardless of score
    = 3NoNEEDS_REVISIONReviewer's verdict is the tiebreaker

    If the judge and reviewer flatly conflict outside this matrix, prefer the reviewer when the disagreement is about code-context nuance the judge couldn't see (it only got the diff, not the live codebase); prefer the judge when it's flagging something the reviewer's own verdict didn't address at all.

  6. Report the verdict with the evidence, not just the label. State the per-dimension scores, any critical violations found, and which action follows — don't just say "judge approved it."

Reference files

  • references/judge-prompt-template.md — the self-contained prompt to send the second-provider CLI, including the requested structured output format.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.