agentleFS
nyldnnyldn/claude-octopus/.claude/skills/skill-refine/SKILL.md

skill-refine

Skill4.4k starsChanged 3 months agoMITView on GitHub

Score and improve an artifact with a frozen rubric, independent models, implemented edits, and bounded rescoring

  • Deletes or force-pushes

3 other projects use a version of this.

Adopted by

Steady: 0 new projects took it in the last 4 weeks, none the 4 weeks before.thin data

Week of Sep 20: 0 new projectsWeek of Sep 27: 0 new projectsWeek of Oct 4: 0 new projectsWeek of Oct 11: 0 new projectsWeek of Oct 18: 0 new projectsWeek of Oct 25: 0 new projectsWeek of Nov 1: 0 new projectsWeek of Nov 8: 0 new projects
Sep 2026Oct 2026Nov 2026
New projects taking a version of this file each week, over the 8 weeks our latest run read: only 0 in all, so this is a hint, not a trend.

Most recently taken up by

Who's talking about nyldn/claude-octopus

The biggest 5 of 53 mentions linking to the repository that our crawl found. Each line is their own headline.

See all 53

Talked about on GitHub, Bluesky and Hacker News (53 times).

On GitHub · 50

45 more on GitHub

On Bluesky · 2

On Hacker News · 1

The file

Show its rules and text

We split this skill into the rules it gives an agent

13 rules. About 0 are common, 9 are distinctive, and 4 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈9

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 4

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Never role-play other models or invent their scores.

    line 11
  2. Distinctive:

    Keep this absolute root in the workflow context and include its assignment in each new shell invocation.

    line 33
  3. Distinctive:

    If the judge adapter or score validator is missing at this root, stop and name the missing component rather than running a command with an …

    line 35
  4. Distinctive:

    Read the supplied artifact and relevant project instructions.

    line 40
  5. Distinctive:

    Before any external provider call, including model-assisted research, inspect the candidate, supporting evidence, and accessible project …

    line 46
  6. Distinctive:

    Ask only for missing information that prevents a useful, bounded run.

    line 63
  7. Very specific:

    Create a unique run directory in the user's project under .claude-octopus/refine/<run-id>/, never in the installed plugin.

    line 75
  8. Very specific:

    Keep the original, best candidate, rubric, model responses, score reports, and an append-only iterations.jsonl owned by …

    line 79
  9. Very specific:

    Use the host's native shell, file inspection, and task tools.

    line 93
  10. Distinctive:

    Before each phase, read its entire resource from the resolved installed plugin root, using the exact phase paths listed below.

    line 97
  11. Distinctive:

    Before external research or judging, inspect candidate, evidence and accessible context for secrets as required above.

    line 108
  12. Very specific:

    Always use refinement-pair.py through its mandatory single-judge adapter.

    line 123
  13. Distinctive:

    Stop after two stalled rounds or any budget, evidence, cancellation or provider blocker.

    line 136

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Rubric-guided refinement

Execute the workflow below when invoked through /octo:refine or /octo:ladder. This is a host-managed workflow using real provider calls. Never role-play other models or invent their scores. The score validator checks structure and arithmetic; it does not verify the truth of a model's judgments.

Set OCTO_SKILL_FILE to the absolute path of this loaded SKILL.md supplied by the host. Resolve its installation before reading resources or running commands. This works without a stable plugin link or trusted startup hooks. If the host supplies no skill path, use its explicit plugin root; stop if neither resolves.

OCTO_ROOT="${CLAUDE_PLUGIN_ROOT:-${CODEX_PLUGIN_ROOT:-${OCTO_ROOT:-}}}"
if [[ -n "${OCTO_SKILL_FILE:-}" ]]; then
  [[ "$OCTO_SKILL_FILE" == /* && -f "$OCTO_SKILL_FILE" ]] || exit 1
  OCTO_ROOT="$(cd "$(dirname "$OCTO_SKILL_FILE")" && pwd -P)"
  while [[ "$OCTO_ROOT" != / && ! -f "$OCTO_ROOT/.claude-plugin/plugin.json" && ! -f "$OCTO_ROOT/.codex-plugin/plugin.json" ]]; do
    OCTO_ROOT="$(dirname "$OCTO_ROOT")"
  done
fi
[[ -n "$OCTO_ROOT" && -f "$OCTO_ROOT/scripts/refinement-judge.sh" && -f "$OCTO_ROOT/scripts/refinement-score.py" && -f "$OCTO_ROOT/scripts/refinement-run.py" && -f "$OCTO_ROOT/skills/skill-refine/SKILL.md" ]] || {
  printf '%s\n' 'Cannot resolve this installed Octopus skill' >&2
  exit 1
}

Keep this absolute root in the workflow context and include its assignment in each new shell invocation. Shell variables may not persist between tool calls. If the judge adapter or score validator is missing at this root, stop and name the missing component rather than running a command with an empty root.

1. Establish scope and budget

Read the supplied artifact and relevant project instructions. Accept text, files, plans, code, images, or other artifacts the host can actually inspect. For binary, visual, or remote artifacts, obtain a faithful readable or rendered snapshot with the appropriate host tools. If you cannot inspect it, stop and name the missing capability. Do not score an unseen attachment or a URL title.

Before any external provider call, including model-assisted research, inspect the candidate, supporting evidence, and accessible project context for credentials and private tokens. Repeat this inspection for new content before dispatch. If secrets are found, stop the run before research or judging and report the affected input or file without quoting its secret values. Do not continue by creating a sanitized copy, changing working directories, or asking a judge to ignore the original. The current adapter restricts writes and tools; it does not enforce isolation from the secret-bearing original or other readable files. Secret-bearing inputs require an execution boundary that prevents those reads, which this adapter does not provide. Host inspection is not an automatic secret scanner or a guarantee of data isolation.

Record the audience, intended outcome, constraints, editable paths, and what must be preserved. A request to improve a plan authorizes editing the plan, not executing the project described by it. A scoring or review-only request produces assessments and proposed revisions without changing the original. Otherwise invocation authorizes scoped improvements to the supplied artifact. Ask only for missing information that prevents a useful, bounded run.

Validate all options before work. Defaults are target 95/100, max five improvement rounds, max 30 elapsed minutes, and max 14 external model calls. --max-rounds accepts integers 1 through 10; minutes and calls must be positive integers; target must be finite and between 0 and 100. Preserve explicit budgets. Retries, research, scoring, and model-assisted edits all consume the call budget. Baseline is round zero. Check remaining time and calls before every dispatch, and reserve two scoring calls before starting an improvement. Bound each call by the remaining elapsed budget. Never reset counters on resume or restart a run to evade a limit. Stop after two rounds without verified improvement.

Create a unique run directory in the user's project under .claude-octopus/refine/<run-id>/, never in the installed plugin. Check the project's ignore rules before writing sensitive content there. If it is not ignored, use a private directory outside the repository and report its path. Keep the original, best candidate, rubric, model responses, score reports, and an append-only iterations.jsonl owned by scripts/refinement-run.py. Never append hand-written iteration decisions. Record hashes, actual model IDs and effort, calls, elapsed time, available usage/cost data, changes, and reasons for keeping or rejecting candidates. Unknown monetary cost stays unknown.

Snapshot the exact initial content and existing dirty edits. Work on a candidate copy. Publish only an accepted candidate into the authorized paths after checking that the original has not changed since it was read. Preserve concurrent edits and stop on a conflict. Never use git reset --hard, git clean, blanket checkout, git add -A, or automatic commits to manage these experiments.

Phase loading and fixed judging contract

Use the host's native shell, file inspection, and task tools. On Codex, load skills/blocks/codex-host-adapter.md from the resolved plugin root. Do not assume that a host exposes a provider or subagent until its tools confirm availability.

Before each phase, read its entire resource from the resolved installed plugin root, using the exact phase paths listed below. These paths work for both Claude and Codex installations. Stop if a resource is missing. Load only the next phase, and reload its resource after compaction before resuming it:

  1. Scope, secret inspection, candidate preservation, and budgets are above.
  2. Load skills/skill-refine/references/phase-2.md to select and freeze the rubric.
  3. Load skills/skill-refine/references/phase-3.md before independent scoring.
  4. Load skills/skill-refine/references/phase-4.md before edits and rescoring.
  5. Load skills/skill-refine/references/phase-5.md before delivery and trajectory.

Before external research or judging, inspect candidate, evidence and accessible context for secrets as required above. Run declared guards before model review. Choose exactly Claude and Codex with preflight.sh --providers claude,codex, without changing the council roster. Preserve fixed explicit models and efforts, record requested and resolved identities, and stop on a required provider failure. A readiness receipt is selection evidence only. Fresh dispatch auth, quota, and policy checks remain mandatory, including when credentials change after selection.

Both independent read-only judges must receive the same frozen candidate, blind rubric, and guard evidence. Hide target, prior scores, author and other judges. Initialize the managed run through refinement-run.py as described in phase 2. Its begin, prepare, and record commands enforce round admission, guard failures, acceptance and two-round stalls. A stopped run cannot be reinitialized or moved to another directory to reset its limits. Keep legacy runs read-only until their evidence has been reconciled; never invent controller state for them. Always use refinement-pair.py through its mandatory single-judge adapter. Reserve both calls before launch and count failures, cancellation, research, retries, and resume against the same total budget and absolute deadline. Never redispatch a reserved pair ID. Raw stdout and one successful seat cannot pass. For a material rubric correction, follow phase 2's explicit version transition and rescore the original before the best candidate within the remaining budget.

Bind assessment identities to actual dispatch metadata, exactly one canonical claude and one codex, plus rubric and candidate hashes. Run scripts/refinement-score.py on both valid assessments. Its conservative lower score, criterion minima, no blockers, passing guards and disagreement limit of 10 points control acceptance. Verify findings before edits. Never average away failure or weaken the rubric. Accept only verified improvement with no regression. Stop after two stalled rounds or any budget, evidence, cancellation or provider blocker. Resume only with matching ledger, original, candidate, current rubric version, fixed panel, models and efforts; never reset counters. Label target success only when the validator reports target_met: true for the delivered bytes. Preserve concurrent edits and existing approval boundaries through delivery.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.