agentleFS
fkguofkguo/nullius/skills/referee-review/SKILL.md

referee-review

Skill20 starsChanged 22 days agoMITView on GitHub

Use when a manuscript or draft needs an independent referee-style assessment before submission, revision, or handoff. Generates a clean-room, offline (no network) referee-review report with Markdown + strict JSON output (generic profile).

The file

We split this skill into the rules it gives an agent

2 rules. About 0 are common, 0 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    A1-A5 gate open: Run nullius integrity-record --approval-id <id> --modes <Mx,...> --notes "..." against that gate's approval_id.

    line 28
  2. Very specific:

    If additional evidence is needed, populate evidence_requests with structured queries (to be executed by an external stage), and keep …

    line 55

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Referee Review (generic, offline)

This skill defines a generic (non-venue-specific) peer-review output contract that is designed to be ingested by an orchestrator as artifacts.

Output contract (must)

  • review.md: first line must be exactly VERDICT: READY|NOT_READY
  • review.json: must validate against schemas/review.schema.json (fail-fast)

Author pre-flight (run before submitting a packet here)

The author agent should walk the M1-M7 pre-approval ritual from skills/research-integrity/ before posting a draft to this reviewer. Findings the author could have caught (hallucinated citation, hallucinated measurement, methodology-not-in-artifacts, frame-lock, etc.) should not be the reviewer's BLOCKING items; this reviewer is for adjudication against the packet, not for catching omissions the author skipped.

Recording the pre-flight. The M1-M7 walk is always required before posting a draft to this reviewer. How the walk is recorded depends on whether an approval gate is open:

  • A1-A5 gate open. Run nullius integrity-record --approval-id <id> --modes <Mx,...> --notes "..." against that gate's approval_id. The receipt is the canonical machine record of the pre-flight and is fail-closed-enforced by nullius approve (missing receipt → INTEGRITY_RECEIPT_REQUIRED, see packages/shared/src/integrity-receipt.ts). This matches the gate-coupled "machine record" path documented in skills/research-integrity/SKILL.md §"Recording the check".
  • No gate open (ad-hoc draft submission). Record the M1-M7 modes you walked inline in your response or in the notebook entry that accompanies the draft — the narrative record from skills/research-integrity/ §"Recording the check" is the audit trail here. Do not invent a synthetic approval_id to feed the CLI: nullius integrity-record is parameterized on approval_id precisely because the receipt is the gate-coupling key, and a junk identifier pollutes .nullius/integrity_log.jsonl without producing any enforcement (this reviewer's pipeline does not consume the log — scripts/run_referee_review.py does not read it).

This reviewer remains generic and non-venue-specific; the pre-flight is the author-side discipline, not a venue requirement.

No-network / role separation

  • Reviewer must not fetch new evidence (no INSPIRE/arXiv/DOI/GitHub/web).
  • If additional evidence is needed, populate evidence_requests with structured queries (to be executed by an external stage), and keep verdict=NOT_READY when blocking.

CLI (deterministic, offline)

python3 scripts/run_referee_review.py --profile generic --packet fixtures/packet.md --out-dir /tmp/review_out

This writes:

  • /tmp/review_out/review.md
  • /tmp/review_out/review.json

Packet format (minimal)

The packet is a Markdown file. Recommended sections:

  • ## Paper diff
  • ## References (short bibliography or notes)
  • ## Artifact pointers (one per line, e.g. - /absolute/path/to/artifact or - file:///absolute/path/to/artifact)

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.