Grade Enrichment

Grade an enriched prompt, goal, or loop spec against its rule pack. This skill is strictly opt-in: run it only when the caller explicitly requests a graded enrichment. The improve skills never chain into it.

Grader selection

Select the grader type per the named tradeoffs in anthropic-agent-eval-grader-types-transcript-grading (see ${CLAUDE_PLUGIN_ROOT}/rules/findings-index.json):

• Code-based (deterministic): for mechanically checkable properties — placeholder tokens present/absent (grep -E '\[[A-Z_]+\]'), character count vs the pack's budget_chars (wc -c; skip this check when budget_chars is null), presence of required structural elements. Cheap, exact, no calibration needed. Always run these first. When the artifact is a file path, run wc/grep on the file; when it is inline text, perform the placeholder scan and character count by direct inspection — never stage inline text through the shell (no heredocs/herestrings).
• LLM-based (rubric judging): for judgment properties — clarity, two-expert unambiguity, right-altitude. Requires calibration (below) before its verdict is trusted.
• Human: recommend when verdicts carry consequences the caller must own, or when calibration is unavailable and the caller declines self-calibration.

Grading procedure

1. Read the matching rule pack from ${CLAUDE_PLUGIN_ROOT}/rules/ for the artifact's kind (prompt, goal, or loop). If the kind is not stated and not obvious from the artifact's shape, ask — never guess.
2. Run every code-based check that applies (placeholders, budget, mandatory loop gates).
3. For the judgment rules, apply LLM-based rubric grading — reason first, then verdict, per rule: does the artifact satisfy the rule, yes or no, and why.
4. Report per-rule verdicts (rule id, pass/fail, one-line reason, the rule's finding_refs) and an overall verdict: pass only when every applicable rule passes.

Calibration gate — graceful, never blocking

Before trusting any LLM-based verdict, read ${CLAUDE_PLUGIN_ROOT}/golden/calibration-record.json.

• Record present and agreement ≥ 0.75 (the 75-90% human-agreement band from llm-as-judge-golden-dataset-calibration; this skill's 0.75 governs — the record's own threshold field is informational): proceed; note the recorded agreement, date, and grader_model in the report. When the current model differs from the recorded grader_model, say so and offer (never require) re-calibration.
• Record missing, unreadable, or below threshold: DO NOT throw, block, or refuse. Report the code-based results normally, mark every LLM-based verdict explicitly as UNCALIBRATED, and offer to self-calibrate: for each case in ${CLAUDE_PLUGIN_ROOT}/golden/{prompt,goal,loop}/, extract only the draft and enriched fields (grep -E '"(draft|enriched)"' <file> — never Read the whole file, which would expose human_verdict and break blindness), grade each blind, then compare against the human verdicts (readable only after all blind verdicts are made), compute agreement, and present the result. If self-calibration agreement is ≥ 0.75, treat this session's LLM verdicts as calibrated for this session only — the result cannot be persisted (this skill has no write access); otherwise recommend human grading. The golden sets ship with the plugin precisely so a fresh install can always do this.

Output format

Report:

1. Grader type(s) selected and why (one line).
2. Calibration status (agreement %, run date — or UNCALIBRATED + the self-calibration offer).
3. Per-rule verdict table: rule id | pass/fail | reason | finding_refs.
4. Overall verdict with the failing rules named, if any.

Critical rules

• Never invoke this skill from an improve skill; grading is the caller's explicit request only.
• Never emit a trusted LLM-based verdict without a calibration record meeting threshold; never block or error out over a missing record either — degrade to UNCALIBRATED + the self-calibration offer.
• Code-based checks are always reported, calibrated or not.
• Cite finding_refs for every rule verdict; resolve shorthands via rules/findings-index.json on request.