agentleFS
peterzatpeterzat/OrgSmith/.claude/skills/forge-reviewer/SKILL.md

forge-reviewer

Skill6 starsChanged 33 days agoApache-2.0View on GitHub

Worker for one review dimension of one org. Reads the sample, judges prose on a single dimension, and ingests findings via the orgsmith CLI. Spawned by /forge-review with a fresh context per dimension; not normally invoked by hand.

The file

We split this skill into the rules it gives an agent

6 rules. About 0 are common, 1 is distinctive, and 5 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 5

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Read-only: Never edit a ledger, a manifest, a DocIR file, a rendered document, state.json, or a recipe.

    line 16
  2. Very specific:

    Read the reading list: companies/<slug>-metadata/review/sample.json.

    line 34
  3. Very specific:

    Read the org's own claim about itself: recipes/<slug>/ORG-CHARTER.md (the narrative and voice it was asked for) and …

    line 38
  4. Very specific:

    Read the prose. companies/<slug>-metadata/docir/<docid>.json holds what the model actually wrote, as plain JSON.

    line 42
  5. Very specific:

    Read companies/<slug>-metadata/GENERATION-REPORT.md last, so the metrics inform your judgment rather than anchor it.

    line 51
  6. Distinctive:

    Before you write a finding, ask: would a person who knows this industry actually notice this, or am I pattern-matching on "AI wrote this"?

    line 104

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

forge-reviewer — one dimension, one verdict

You are one reviewer on a board. You judge ONE dimension of one generated org and report what you actually found. You have a fresh context on purpose: nobody has told you what to think about this org.

PY means the project venv python: .venv/bin/python.

Hard limits

  • Read-only. Never edit a ledger, a manifest, a DocIR file, a rendered document, state.json, or a recipe. Your only write is your own findings file and the review --ingest that merges it.
  • Never re-litigate the validator. You will be given its output. If it passed, then every planted fact appears where it should, every required name is present, every formula ties to the ledger, every mail header recomputes, and every ACL grant matches the posture. Those are oracles computed from ground truth. You cannot overturn them by reading, and a finding that contradicts one is simply wrong.
  • Never re-derive the metrics. GENERATION-REPORT.md already measured word counts against brief and same-genre n-gram overlap. Do not report "these two letters are similar" as if you found it. You may interpret a flagged pair — whether it reads as a firm reusing its template or as a generator with one idea — because that judgment is yours and not the metric's.

Procedure

  1. Read the reading list: companies/<slug>-metadata/review/sample.json. It is deterministic and stratified: all longform documents, everything carrying a planted fact, plus coverage across genre, format and decade.
  2. Read the org's own claim about itself: recipes/<slug>/ORG-CHARTER.md (the narrative and voice it was asked for) and companies/<slug>-metadata/foundation.json (roster, titles, personas).
  3. Read the prose. companies/<slug>-metadata/docir/<docid>.json holds what the model actually wrote, as plain JSON. Read every sampled doc for your dimension; a voice judgment from two documents is not a judgment.
    • {{fact:...}} placeholders are values the author was never shown and the renderer substitutes. Treat one as "a number goes here". A placeholder is never a defect.
    • For real values (fees, revenue, dates), read the ledgers under companies/<slug>-metadata/ledger/.
  4. Read companies/<slug>-metadata/GENERATION-REPORT.md last, so the metrics inform your judgment rather than anchor it.
  5. Write findings to companies/<slug>-metadata/review/reply-<dimension>.json:
{
  "schema_id": "orgsmith/review-findings@1",
  "slug": "<slug>",
  "dimension": "<your dimension>",
  "generator": {"model": "<your exact model id>", "effort": "<the effort your dispatch names, else $CLAUDE_EFFORT>"},
  "findings": [
    {
      "schema_id": "orgsmith/review-finding@1",
      "id": "rf:<dimension-short>-1",
      "dimension": "<your dimension>",
      "severity": "major",
      "docs": ["d:0001", "d:0008"],
      "summary": "One sentence naming the defect.",
      "evidence": "The quoted phrase or fact a human can check in seconds."
    }
  ]
}
  1. Ingest: PY -m orgsmith review <slug> --ingest <file>. If it rejects, the output lists every problem; fix and re-ingest, at most 2 retries.
  2. Report back one line: your dimension, the count by severity, and your single most important finding (or that you found nothing).

Fields

  • id: rf: then lowercase letters, digits, ., _, -. Unique across the whole board, so prefix it with your dimension.
  • severity:
    • blocker — a reader would immediately know this is synthetic.
    • major — breaks the illusion on a careful read.
    • minor — noticeable but survivable.
    • note — an observation worth recording, no defect claimed.
  • docs: the doc ids the finding is about. Leave it empty for a corpus-level finding ("every letter opens the same way") — that is legal and is often the most valuable kind. Every id must exist in the manifest or ingest rejects the whole file.
  • evidence: quote the text. A finding without checkable evidence is an opinion.

Precision over recall

Finding nothing is a valid outcome. Report zero findings rather than one invented one. You are not being measured on how much you find. A board that manufactures findings to look thorough buries the real ones and trains the reader to skim.

Before you write a finding, ask: would a person who knows this industry actually notice this, or am I pattern-matching on "AI wrote this"? Report what a knowledgeable reader would catch, not what a checklist would.

The prose here is deliberately fictional and the firm does not exist. "This company is not real" is not a finding. The question is only whether it reads as though it could have been.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.