hillclimb
Skill1 starsChanged 3 days agoMITView on GitHub
Improve one skill in this repo against its eval cases in evals/<skill>/, one change per round, keeping a change only when train and test scores rise beyond noise. Use when asked to hillclimb, tune or improve a skill with its evals, or to cut a skill's cost at the same score.
15 other projects use a version of this.
Adopted by
Most recently taken up by
- niilohlin/skillstheir own version, first seen Nov 16, 2026
- markjoshwel/skillstheir own version, first seen Nov 16, 2026
- adelrodriguez/skillstheir own version, first seen Nov 15, 2026
- Yeachan-Heo/oh-my-claude-sisyphustheir own version, first seen Nov 8, 2026
- Suge8/friedbuntheir own version, first seen Nov 8, 2026
- Yeachan-Heo/oh-my-claudecodetheir own version, first seen Nov 8, 2026
Used with
Projects with it also use babysit-pr, retro and 1 more.
Other files people use alongside this one.
Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
owidSkillbabysit-pr
Spawn a background agent that babysits an open PR — triggers a Codex review, watches CI, judges and fixes valid findings, replies to and resolves review threads, and loops to a cap. Never merges. Trigger when the user asks to "babysit the PR", "ask for codex review(s) and watch CI", or after pushing a substantial chunk to a PR branch.
Also in 3 projects that use this
AWSSkillretro
Conduct a retrospective on a coding session.
Also in 4 projects that use this
GoogleSkillgrill-me
A relentless interview to sharpen a plan or design.
Also in 4 projects that use this
Other versions15 projects use a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 15
- Yeachan-Heo/oh-my-claudecode★ 41ksame name
- Yeachan-Heo/oh-my-claude-sisyphus★ 41ksame name
- rebase-energy/hillclimb★ 37same name
- Suge8/friedbun★ 13same name
- michielvandermeer/skills★ 9same name
- zombico/mojulo★ 4same name
- jonmumm/skills★ 3same name
- paureis/pau-skills★ 2same name
- adelrodriguez/skills★ 1same name
- Astatide1337/Skills★ 1same name
5 more not listed
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
The file
We split this skill into the rules it gives an agent
9 rules. 0 are common, 4 are distinctive, and 5 are very specific to this project.
0
Common
Most of the 14 edited versions we read keep these.
4
Distinctive
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 14
Never put eval content into the skill: No case topics, numbers, file names, wording or answers in SKILL.md.
- Distinctive: line 16
Do not edit cases or graders mid-climb: If one is wrong, stop and say so: changing the ruler resets the baseline.
- Very specific: line 17
Prefer making an existing instruction clearer or removing one over adding text.
- Distinctive: line 23
Check the ruler before climbing:
- Distinctive: line 26
If the suite is above 0.95, report that it is saturated and stop: it needs harder cases, not a better skill.
- Very specific: line 31
Read the train misses: grades.json in evals/results/<run>/train/*/run*/, then the run's files under home/ and its trace.jsonl to see why.
- Very specific: line 35
Keep the edit if compare says Keep.
- Very specific: line 36
Add an entry to evals/<skill>/hillclimb-log.md: date, root cause, the edit in one line, train and test before and after with bands, kept or …
- Distinctive: line 38
Stop after the round limit (default 5), or after 3 rounds in a row with nothing kept.
“Common” is measured: kept by at least 6 in 10 of the 14 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Hillclimb
Improve plugins/<plugin>/skills/<skill>/SKILL.md against evals/<skill>/. Read evals/README.md first for how runs and grading work. If it says the skill's evals are paused, stop and say so: a run would cost money and grade nothing.
Rules that keep the score honest
- Never read the test set. Not
evals/<skill>/test/, notevals/results/*/test/.run.pyprints only the test aggregate; that is all you get. - Never put eval content into the skill. No case topics, numbers, file names, wording or answers in SKILL.md. A change must be a general rule a new learner on a new topic would benefit from.
- One change per round, aimed at a root cause, so a score change can be traced to it. Rewording without a reason is not a change.
- Do not edit cases or graders mid-climb. If one is wrong, stop and say so: changing the ruler resets the baseline.
- Prefer making an existing instruction clearer or removing one over adding text. SKILL.md length is a cost every run pays.
Setup
- Goal: the user's, or score by default. For cost: same score with a cheaper
--modelor a shorter skill. - Baseline: the newest full run of the current skill in
evals/results/(theskill_sha256in itssummary.json, a hash of every file in the skill folder, matches the current one), elseevals/run.py run <skill>. - Check the ruler before climbing:
evals/run.py rejudge <baseline>. If more than 1 in 10 verdicts flip, the claims are too vague: report which, and stop.- A case that scores 1.0 on every run has no headroom; one that swings widely between runs is noise. Note both.
- If the suite is above 0.95, report that it is saturated and stop: it needs harder cases, not a better skill.
- With fewer than 5 test cases, a real fix rarely moves test beyond noise, so most rounds will end Unclear. Say so before starting.
Each round
- Read the train misses:
grades.jsoninevals/results/<run>/train/*/run*/, then the run's files underhome/and itstrace.jsonlto see why. Look across cases for the same cause. - Name the root cause in one sentence, as a gap or ambiguity in SKILL.md, not as a fact about a case.
- Make one edit to SKILL.md.
evals/run.py run <skill>, thenevals/run.py compare <previous kept run> <new run>.- Keep the edit if compare says Keep. If it says Unclear, keep only when the edit is plainly general; three Unclear keeps in a row is overfitting: revert to the last clear Keep. Otherwise
git checkout plugins/<plugin>/skills/<skill>/SKILL.md. - Add an entry to
evals/<skill>/hillclimb-log.md: date, root cause, the edit in one line, train and test before and after with bands, kept or reverted.
Stop after the round limit (default 5), or after 3 rounds in a row with nothing kept. On a stall, edit nothing: sort each remaining train miss into skill gap, bad case, bad grader or noise, with the evidence, and report.
Finish
Report the start and end scores for train and test with their bands, and each kept edit. Recommend merging only if test rose beyond noise. Merging a SKILL.md change means bumping the version in each of the plugin's manifests and running scripts/check.sh. Do not commit unless asked.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

