agentleFS
sxntixgosxntixgo/frugal-skills/skills/skill-reviewer/SKILL.md

skill-reviewer

Skill0 starsChanged 4 months agoMITView on GitHub

Reviews a Claude Code skill against the repo's 8 authoring rules and proves it works via a baseline eval comparison (with-skill vs. disabled). Use when a skill was just created or edited, or when asked to review, evaluate, or benchmark a skill.

24 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use skill-builder.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • BlockSkill

    skill-builder

    Create, edit, or inspect Berd skills stored as skill folders with SKILL.md files. Use when the user wants to build a new skill, update an existing skill, convert a workflow into a reusable skill, or decide whether instructions, scripts, references, or assets belong in a skill.

    Also in 3 projects that use this

The file

We split this skill into the rules it gives an agent

3 rules. 0 are common, 1 is distinctive, and 2 are very specific to this project.

  • 0

    Common

    Most of the 19 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Run a baseline eval comparison (with-skill vs.

    line 19
  2. Very specific:

    Create a TodoWrite item per stage.

    line 29
  3. Very specific:

    Read the skill's SKILL.md, references, scripts, and evals/evals.json.

    line 36

“Common” is measured: kept by at least 6 in 10 of the 19 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Skill Reviewer

Judge a skill. skill-creator builds; you review. Always run the static rubric, then always proceed toward the baseline evals — but STOP for approval before spawning any isolated runs, because they cost tokens and time.

The 8 authoring rules this rubric scores against are detailed in references/rubric.md. (When working inside this repo, the same rules are also documented in CLAUDE.md.)

Scope

This skill DOES:

  • Score a skill against rules 1–8 (static rubric).
  • Run a baseline eval comparison (with-skill vs. without-skill), or a blind A/B vs. the previous version for an edit.
  • Render a PASS / ITERATE verdict with the top fixes.

This skill does NOT:

  • Rewrite the skill → that's skill-creator. Report findings; the author fixes.
  • Spawn any isolated run before the user approves at the Stage B gate.

Workflow

Create a TodoWrite item per stage. Checkpoint the Stage A rubric findings to ./.claude-skills-work/<skill>/review/ so a review can resume if interrupted. (The Stage B eval RUNS use a separate workspace, ./.claude-skills-work/evals/<skill>/iteration-N/ — see references/running-evals.md. The two paths are intentionally distinct: review findings vs. generated runs.)

Stage A — Static rubric (always)

Read the skill's SKILL.md, references, scripts, and evals/evals.json. Score it against all 8 rules using references/rubric.md. Rate each finding: 🔴 Must fix · 🟡 Should fix · 🟢 Consider. Write findings to review/rubric.md and report them. If rule 8 fails (no evals/evals.json), say so — Stage B can't run without cases.

Stage B — Baseline evals ⛔ STOP GATE

  1. Count the work: cases × 2 = runs (with_skill + without_skill). For an EDIT, the baseline is a snapshot of the previous version instead of "no skill".
  2. Report the run count and that each run is a fresh isolated context with its own token/time cost, then STOP and ask the user to approve before spawning anything.
  3. On approval, follow references/running-evals.md: spawn one subagent per run from a clean context, save outputs + a timing.json (tokens, duration) per run, grade each case against its assertions (or expected_output if assertions aren't written yet), and aggregate a workspace benchmark.json with with_skill, without_skill, and delta.

Stage C — Verdict

Combine the rubric findings and the benchmark delta into a verdict:

  • PASS — no 🔴 findings and the skill measurably helps (positive pass-rate delta at acceptable token/time cost).
  • ITERATE — list the top fixes (🔴 first) and which eval cases regressed or failed, so skill-creator can address them.

Red flags (stop and correct)

If you notice…Do this
About to spawn runs without approvalSTOP — the Stage B gate is mandatory
Editing the skill yourselfDon't — report findings; skill-creator fixes
Passing a skill on the rubric aloneA PASS also needs a positive eval delta
Grading a PASS on a vague assertionRequire concrete evidence; quote the output
Skipping Stage A because "evals will catch it"Rubric is always run first
No evals/evals.json presentReport rule 8 failure; ask author to add cases

Done when

  • Rubric findings are written and reported.
  • Either the baseline benchmark exists with a delta, or the user declined the eval run at the gate (note that in the verdict).
  • A PASS / ITERATE verdict is given with the top fixes.

References (load only when needed)

  • references/rubric.md — Stage A: the 8-rule checks + severity guidance.
  • references/running-evals.md — Stage B: spawning runs, grading, aggregating.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.