agentleFS
NilsHellwigNilsHellwig/acl-paper-skills/skills/reviewer/SKILL.md

reviewer

Skill3 starsChanged 14 days agoMITView on GitHub

Simulated ARR review of the paper along the current official review form, plus a guidelines check. Writes review.md and report items.

38 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use tester, planner and 3 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • KJ-devsSkill

    tester

    Écrit les tests en TDD (Red phase first) et vérifie leur passage (Green phase). Frontend avec Playwright, backend avec BDD Given-When-Then.

    Also in 4 projects that use this

  • am-willSkill

    planner

    Create comprehensive, phased implementation plans with sprints and atomic tasks. Use when user says: "make a plan", "create a plan", "plan this out", "plan the implementation", "help me plan", "design a plan", "draft a plan", "write a plan", "outline the steps", "break this down into tasks", "what's the plan for", or any similar planning request. Also triggers on explicit "/planner" or "/plan" commands.

    Also in 5 projects that use this

  • tesla0225Skill

    orchestrator

    Issue/要件を受けたら分解→担当割当→実装→検証→統合を回す手順。未整備なら /repo-init でbootstrap。

    Also in 4 projects that use this

  • Harut8Skill

    architect

    Also in 3 projects that use this

  • NVIDIASkill

    research

    Before designing a feature, research how comparable libraries solve the problem, check best practices, known pitfalls, and upstream changes. Use when starting a new feature, investigating an unfamiliar area, or the user asks to research something.

    Also in 3 projects that use this

The file

We split this skill into the rules it gives an agent

1 rule. 0 are common, 1 is distinctive, and 0 are very specific to this project.

  • 0

    Common

    Most of the 29 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Read the whole paper in reading order, including appendices, once without writing.

    line 32

“Common” is measured: kept by at least 6 in 10 of the 29 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

This skill is ARR-only. When paper.config.json has venue semeval, say that SemEval has no ARR review form and stop.

Review the paper as an ARR reviewer would, so the authors see the weaknesses before real reviewers do. The review form and reviewer guidelines change between cycles; every run fetches them.

PK means python3 ${CLAUDE_PLUGIN_ROOT}/scripts/paperkit.py, run from the project root.

1. Fetch the form

PK guidelines reviewer

It downloads the ARR review form and reviewer guidelines into a temp folder. On ⚠️: tell the user what is missing and stop, since a review against a remembered form would mislead. Read both in full: the form fields, the score scales with their descriptions, and what the guidelines tell reviewers to do and to avoid (the guidelines list common unhelpful review patterns; a useful review avoids them too).

2. Context

Ask with AskUserQuestion what changes the review and is not in the project: the track or area the paper is submitted to, long or short, and whether it is a resubmission. For a resubmission, ask for the previous reviews and meta-review (offer reviews/): a reviewer of a resubmission checks whether the earlier weaknesses were addressed, so the review must do the same.

3. Compliance first

Load and run the guidelines-check skill, carrying over the answers from step 2 so it asks only what is still open. Its findings (including the Responsible NLP checklist) go to report.md as usual and inform the review's soundness and ethics fields.

4. Read like a reviewer

Read the whole paper in reading order, including appendices, once without writing. Then review it:

  • What is the claim, and does the evidence carry it? Baselines, ablations, statistical significance, data splits, contamination, number of runs.
  • Is it reproducible from the paper (hyperparameters, prompts, data, code)?
  • Related work: is the closest prior work present? Search with the acl-anthology and arxiv skills for work the paper should compare to, and read what you name (the literature skill) before saying the paper misses it.
  • Clarity: could a reader in the area reimplement it?

Apply the persona or focus from the argument if one was given; otherwise review as a competent, fair reviewer from the paper's area.

5. Write review.md

Fill every field of the fetched form in review.md in the project root, in the form's order and with its field names: summaries, strengths, weaknesses, comments, every score with a one-line justification using the scale's own wording. Each weakness names the place in the paper and what would fix it. Where a weakness rests on a reviewer guideline or a form criterion, name its identifier and the document's URL as PK guidelines prints it. Overwrite an earlier review.md only after asking; otherwise number it (review-2.md).

6. Report items

Each weakness that the authors can fix before submission becomes an item in report.md, category Review, severity by how much it would move the score, with Rule: where the weakness rests on a guideline. Weaknesses that need new experiments are 🔴 or 🟠 with the experiment named.

Report in chat: the overall assessment with its score, the three weaknesses most likely to sink the paper, and what fixes them.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.