agentleFS
udaysharmadevudaysharmadev/Skills/skills/referee/SKILL.md

referee

Skill12 starsChanged 56 days agoMITView on GitHub

Reviews a change against intent, design, correctness, complexity, security, tests, maintainability, and repository conventions. Use for code, diffs, pull requests, or implementation reviews that should produce only evidence-backed actionable findings.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

11 rules. About 0 are common, 9 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈9

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    If intent remains unknown, continue a bounded correctness review and name that gap.

    line 18
  2. Distinctive:

    Read references/review-protocol.md for an impactful change, incomplete evidence, or uncertain finding.

    line 39
  3. Very specific:

    No subagents, or trivial diff → structured self-review per references/finding-format.md (complete diff, criteria re-read, hunt what you'd …

    line 46
  4. Very specific:

    Structured self-review: no subagent available. The author reviewing own work is attached to it, so the checklist in …

    line 73
  5. Distinctive:

    Create a coverage note: files and commits reviewed, criteria, repository instructions, tests and commands observed, public or operational …

    line 88
  6. Distinctive:

    Only actionable items: each with location, problem, why it matters, and a fix direction (see format reference).

    line 114
  7. Distinctive:

    approve with findings: only findings explicitly safe to fix forward.

    line 127
  8. Distinctive:

    If a command, fixture, caller search, or environment is unavailable, record the gap and narrow the scope or verdict rather than guessing.

    line 142
  9. Distinctive:

    Every finding: file:line location + why it matters + fix direction.

    line 156
  10. Distinctive:

    Read references/research.md when a decision depends on an external standard, a numerical claim, or a fast-moving practice.

    line 207
  11. Distinctive:

    Read references/finding-format.md for severity, test, security, and self-review discipline.

    line 210

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

referee: review like you didn't write it

Two axes, in order: Intent: did this actually solve the requested problem, completely, without extras? Then quality: is it correct, maintainable, safe, proportionate? A beautiful implementation of the wrong thing fails. Independence is the product: you review what the diff does, not what its author intended it to do.

Prerequisites

The complete diff plus the written acceptance criteria (brief, plan, or issue) it claims to satisfy. Recover missing criteria from the user request, issue, or repository first. If intent remains unknown, continue a bounded correctness review and name that gap; request only the evidence needed for an intent verdict.

Authority and review scopes

Review scope (what is under review) and review staffing (who reviews) are orthogonal: pick one scope from the table, one staffing from Review staffing, and disclose both.

Review is evidence and advice; it does not authorize edits, merges, deployments, or policy changes. Inspect the relevant repository state, but label source, runtime, and test surfaces not examined. A clean diff is not proof that the changed behavior was exercised.

ScopeUse whenResult
Diff reviewbounded change and criteria availableintent and quality verdict with findings
Change-impact reviewpublic, data, auth, concurrency, or operational surface changedcaller, consumer, migration, and rollback map
Pre-merge reviewchange is ready for decisionverdict plus verification gaps and release risk
Review recoverydiff or criteria incompleteexact missing evidence; no simulated verdict

Read references/review-protocol.md for an impactful change, incomplete evidence, or uncertain finding.

Tool selection/fallback

  • Subagent support + non-trivial change → fresh-context review (diff + criteria only, no author commentary); disclose the mode.
  • No subagents, or trivial diff → structured self-review per references/finding-format.md (complete diff, criteria re-read, hunt what you'd flag in someone else's code); disclose plus its weakness.
  • Neither time nor context for the full pass → decline to skim: state the minimum needed (split, brief, criteria) rather than fake depth.

When to use

  • "review my diff before I merge"
  • "check whether this change actually implements the spec"
  • "review this but only flag what truly blocks, skip the nits"

When NOT to use

  • Mid-slice debugging → sleuth owns the live loop; review comes when the diff exists.
  • Architecture decisions → headroom; test strategy → proof.
  • The author is asking questions ("is this approach reasonable?"), that's a conversation, not a review; answer it.

Review staffing (disclose which ran)

Fresh-context (preferred): a subagent receives the diff + the brief/plan/acceptance criteria: nothing else. No author commentary, no "here's why I did it". If subagents are available and the change is non-trivial, use this; say review type: fresh-context.

Structured self-review: no subagent available. The author reviewing own work is attached to it, so the checklist in references/finding-format.md runs mechanically: wait for the complete diff, review against the written acceptance criteria (not memory of them), and hunt specifically for what you'd flag in someone else's code. Say review type: self-review: the disclosure is mandatory.

Workflow

1. Frame

What was requested (brief/plan/issue), what the diff claims to do, scope (size, files, risk areas). If the diff claims things it doesn't contain or contains things never requested: that's already a finding.

Create a coverage note: files and commits reviewed, criteria, repository instructions, tests and commands observed, public or operational surfaces considered, and exclusions. This prevents a broad verdict hiding a narrow pass.

2. Intent pass

Against the acceptance criteria, one by one: solved, partially solved, or missing? Anything in the diff that serves no criterion (scope creep, drive-by refactors, dead code introduced)? The intent pass produces blockers faster than any quality check.

3. Quality pass

Walk the diff: correctness (logic, edge cases, error paths, concurrency), tests (exist? right boundary? would they catch a revert?), security (input handling, authz on new surfaces), performance (obvious only, hotpath owns measurement), conventions (project style wins over taste), complexity (could this be simpler without losing anything?).

For changed contracts, trace representative producer and consumer paths rather than reviewing a signature in isolation. Check configuration, serialization, migration, authorization, retry, cache, and observability implications only when the diff touches those mechanisms.

4. Findings

Only actionable items: each with location, problem, why it matters, and a fix direction (see format reference). No "consider...", no style opinions with a rule behind them already, no praise padding. Findings you can't act on get deleted, not softened.

Tie every finding to a violated criterion, observable failure mode, repository convention, or supported invariant. If evidence is incomplete, name what would confirm it and calibrate confidence. Do not call a possible concern a blocker merely to appear cautious.

5. Verdict

  • approve: nothing above minor;
  • approve with findings: only findings explicitly safe to fix forward;
  • request changes: any major that should not merge as-is, or any blocker.

Nits never affect the verdict. State which mode ran, the diff scope covered, and anything explicitly not reviewed ("docs/" skipped).

Override rule: the verdict binds until a human explicitly overrides it. approve with findings and non-blocker request changes may be overridden by the human with a recorded reason; a blocker verdict requires re-review after the fix, never a silent override. Every report ends with an override: line naming who may overrule this verdict and what that requires.

Review failure handling

If a command, fixture, caller search, or environment is unavailable, record the gap and narrow the scope or verdict rather than guessing. If evidence disproves a finding, remove it. If one root cause produces several symptoms, report one primary finding with affected locations so the author can fix the system rather than chase individual warnings.

Anti-Patterns (The Banned List)

  • Context Starvation (Isolation Review): reviewing a file in a vacuum without checking its callers. If a function signature changes, you must use codebase search tools to verify that all upstream callers were updated. A review that misses broken downstream systems is a failed review.
  • LGTM Syndrome (Rubber-Stamping): approving AI-generated code just because it is syntactically clean and has no linter errors. AI code often looks perfect but solves the wrong problem. The Intent Pass must break this illusion of safety.
  • Nit-Picking / AI Pedantry: filling a review with 15 stylistic opinions that don't affect correctness or performance. Keep the noise down; nits are collapsed.

Quality gates

  • Every finding: file:line location + why it matters + fix direction.
  • Intent checked criterion-by-criterion against the actual written criteria.
  • Severity honest: a preference is a nit, not a major; severity inflation is how reviews get ignored.
  • Review type disclosed; self-review explicitly names its weakness.
  • Zero non-actionable findings.
  • Verdict scope matches evidence: approve means no blocker in reviewed scope, not that an unreviewed runtime or deployment is safe.
  • Contract-changing diffs have consumer/caller or migration evidence, or the report names why that evidence was unavailable.
  • Findings are grouped by root cause and confidence matches the evidence.

Stop conditions

  • Verdict delivered → done.
  • Diff unreviewably large or context-free → say what's needed (split, brief, running app), don't skim and pretend.
  • Findings reveal the change solves the wrong problem → verdict request changes with the intent finding; route back to distill/masterplan.
  • Evidence needed for a safe verdict unavailable → return review recovery with exact missing diff, criteria, output, or environment.
  • Request asks the reviewer to merge, deploy, or mutate state → keep review separate and await explicit operational authorization.

Output contract

review type: fresh-context | self-review
scope: 14 files, +312/−87: auth middleware + tests

verdict: request changes

| Sev | Location | Finding | Fix direction |
| --- | --- | --- | --- |
| blocker | auth.ts:41 | role check happens after query: data leaks for !owner | check ownership in the query WHERE |
| major | auth.test.ts | no test for revoked-token path | add API-level test |
| nit | naming.ts:8 | `d` → `durationMs` | rename |

intent: 4/5 criteria met: "session revocation" missing (see major)
not reviewed: public/ assets
override: human may overrule majors with reason; blockers need re-review

Findings table is the whole report; no essay, no restating the diff. For an impactful change, append review coverage and verification gaps beneath the table in terse bullets.

Research basis

Read references/research.md when a decision depends on an external standard, a numerical claim, or a fast-moving practice. The ledger records what the source supports, what it does not support, and when to reverify. Read references/finding-format.md for severity, test, security, and self-review discipline. Read references/review-protocol.md for impact mapping, coverage records, or evidence gaps.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.