evolving-with-swarms
Skill398 starsChanged 8 months agoMITView on GitHub
Improves a program against an automatic scorer by running rounds of cheap Haiku 5.5 swarms, where each proposer suggests one short idea and one writer per swarm turns the ideas into a new program that is scored and kept in an archive, all metered to a dollar cap through headless claude -p. Use for "evolve this program against my evaluator", "optimize this function's score cheaply", "OpenEvolve-style search on a budget", "AlphaEvolve-style construction search with Haiku", "run a swarm to maximize this score", "search for a better heuristic or construction", or any program whose quality a script can score, when the budget is small. Ships the driver, a gateway with a dollar cap and a usage-window pause, a sandbox hook for model-written code, an example problem and an offline selftest; runs from Claude Code on the web or the desktop.
In this repository117 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- accessing-github-reposSkill
- agent-routingSkill
- animating-with-codeSkill
- api-credentialsSkill
- asking-questionsSkill
- assessing-impactSkill
- atprotoingSkill
- bm25Skill
- browsing-blueskySkill
- building-github-indexSkill
- building-github-index-v2Skill
- capturing-browser-logsSkill
- categorizing-bsky-accountsSkill
- challengingSkill
- chartingSkill
- charting-vega-liteSkill
- check-toolsSkill
- cloning-projectSkill
- closing-issuesSkill
- coding-mojoSkill
- compiling-wgslSkill
- composing-htmlSkill
- configuringSkill
- container-layerSkill
- controlling-spotifySkill
- convening-expertsSkill
- converting-filesSkill
- crafting-instructionsSkill
- creating-bookmarkletsSkill
- creating-kbSkill
- creating-mcp-serversSkill
- creating-skillSkill
- creating-videoSkill
- curating-bsky-listSkill
- deciding-with-confidenceSkill
- declaring-invariantsSkill
- declaudingSkill
- delegating-with-contextSkill
- describing-productsSkill
- detecting-tips-zonesSkill
- developing-preactSkill
- dispatching-subagentsSkill
- down-skillingSkill
- drawing-videoSkill
- drawing-with-pycairoSkill
- dunking-anonymouslySkill
- exploring-codebasesSkill
- exploring-dataSkill
- extracting-keywordsSkill
- featuringSkill
- fetching-blocked-urlsSkill
- finding-skillsSkill
- flowingSkill
- forecasting-reversoSkill
- gatingSkill
- generating-patchesSkill
- generative-thinkingSkill
- githubbingSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 36
Use the swarm when the budget is small or the goal is "good, cheaply".
- Distinctive: line 94
No sandbox only under the earned exception above.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Evolving with swarms
Each round, scripts/evolve.py runs K swarms in parallel. A swarm is N proposer calls,
each reading one parent program and one focus x thinking-style prompt and replying with
an idea of at most 45 words, then one writer call that turns the ideas and the parents
into a complete program. The program is scored by your evaluator as soon as it exists
and joins the archive the next round draws parents from: refine one of the best two,
combine three of the best ten, explore two from anywhere, start fresh from the initial
program. Every model call goes through headless claude -p, is priced at list rates
from its modelUsage, and counts against a dollar cap.
The design is a text-only version of SwarmEvolve (Jiang and Karniadakis, arXiv 2611.08483), whose agents think inside an open model's activations, which an API model does not expose.
When it pays
Measured 2026-11-09 against stock OpenEvolve on Sonnet 5.5, three seeds per arm, three problems (references/evidence-2026-11.md):
| swarm (defaults) | Sonnet loop, best effort per problem | |
| $ to 99% of reference, circle packing | $0.013–0.019 | $0.32–0.44 |
| $ to 99%, hexagon packing | $0.035–0.060 | $0.53–0.58 |
| $ to 99%, Heilbronn triangle | $0.09–0.50 | $0.77–1.02 |
| best at $0.25 | above every paired Sonnet seed on all three | |
| final score at $3 | swarm stops at 60 min, about $1 | the same finals, in 7–42 minutes |
Use the swarm when the budget is small or the goal is "good, cheaply". When the last 0.1% matters, the money is there, and wall-clock time counts, a Sonnet loop at the problem's best effort level gets to the same place faster.
When NOT to use this skill
| situation | use instead |
| hundreds of failing tests to drive to green | swarming-to-green |
| choosing a model or effort for one subagent or a cascade | agent-routing |
| many independent tasks with checkable outputs, not one program to improve | dispatching-subagents (headless runner) |
| quality only a person or a judge model can rate | no scorer, no search: write the scorer first, or use a judged loop |
claude.ai chat, where there is no claude binary | orchestrating-chat-agents |
Abandon a run mid-way when:
statusshowsrejected_above_maxclimbing or a best score above anything the problem allows: the scorer has a hole and the search is now optimising the hole;- the best score is flat for ten rounds and
rounds.jsonlshows every refine swarm parenting the same program: the archive has converged, and more rounds buy nothing; last_quota.seven_daypasses about 0.40: the gateway stops at 0.45 anyway.
Earned exceptions
| rule | banned when | earned when |
| writer at effort medium | — | effort low on a problem already known to be easy (circle packing: low reached 0.999 cheaper) |
| N=8 proposers per swarm | the problem is new and N=0 has not been tried | an N=0 control on this problem lost to N=8 (on circle packing the writer alone nearly tied) |
| no sandbox prefix | the evaluator imports model-written code on a machine with secrets or network | a throwaway container with no credentials, and the user agreed |
Setup
- Make a problem directory. Copy
assets/example/and replace its four files:problem.md: the task in plain words, including the reference value if known;initial_program.py: a valid, weak starting program;evaluator.py:evaluate(program_path) -> dictof numbers and strings, with the score underscore_key(defaultcombined_score) and anerrorkey on failure;config.toml: entry function name, the interface text the models see, budgets.
- Score the initial program by hand and read the dict:
A score of 0 with nopython3 -c "import importlib.util as u; s=u.spec_from_file_location('e','evaluator.py'); e=u.module_from_spec(s); s.loader.exec_module(e); print(e.evaluate('initial_program.py'))"error, or a missingscore_key, means the contract is wrong; fix it before spending anything. - Set
score_maxfrom a bound you can prove (a known optimum, an area or volume bound), slightly above it. Scores above it count as 0 and are flagged. Leave it unset only when no bound exists, and then re-check the best program independently. - Choose the sandbox. The evaluator imports and runs model-written code.
- Claude Code on the web with the claude-workspace hub: the hub jail.
and put[sandbox] prefix = ["sh", "/home/user/claude-workspace/scripts/jail.sh", "-C", "{run_dir}", "--"] chown_uid = 65534--run-dirunder/tmp/jail/<name>/(mkdir andchmod 755it first). Launch through a script file: the hub's jail_route guard refuses a Bash command that names/tmp/jail/next topython3. - Desktop with Docker:
prefix = ["docker", "run", "--rm", "--network=none", "-v", "{run_dir}:{run_dir}", "python:3.12-slim"](add numpy or what the evaluator needs to the image). - No sandbox only under the earned exception above.
- Claude Code on the web with the claude-workspace hub: the hub jail.
- Check the CLI:
which claude(desktop: the app's bundled CLI may not be on PATH; setEVOLVE_CLAUDE_BIN). Then run the offline selftest:python3 scripts/test_evolve.py # three "ok" lines, about 3 s, no model calls
Run
S=/mnt/skills/user/evolving-with-swarms/scripts
python3 $S/evolve.py run problem/config.toml --run-dir RUN --seed 1 --cap 1.0 --wall 60 > RUN.log 2>&1Launch it with Bash run_in_background: true and end the turn. The completion line is
done -> RUN/summary.json; relaunching the same command resumes from RUN/archive.json
and keeps the spent dollars. Check in with python3 $S/evolve.py status RUN. In Claude
Code on the web a background task is stopped after two hours: keep --wall under 110
minutes per launch, or relaunch on the completion notice.
knob ([swarm], [budget]) | default | measured |
K, N | 4, 8 | K=2,N=16 and K=8,N=4 did no better on circle packing; N=0 nearly tied |
proposer_effort | low | the floor; Haiku still thinks on about a third of calls |
writer_effort | medium | low helped on circle packing, hurt on hexagon and Heilbronn |
writer_model | haiku | sonnet (one seed): 99% at $0.11 on circle packing, between swarm and loop |
cap_usd, wall_minutes | 1.0, 60 | the swarm spent $0.9–1.4 an hour on 4 vCPUs |
On a new problem, run the default and an N=0 control at the same cap before a long run.
Reading results
RUN/summary.json: programs, best_score, spent, usd_to (dollars when a program
first scored 0.99, 0.999, 0.9999), calls_by_role, rejected_above_max, duplicates,
no_code, last_quota. RUN/best_program.py is the top program. events.jsonl has one
line per evaluation with the spend at that moment, which is the curve to plot;
rounds.jsonl has each swarm's parents and ideas.
Common failure modes
best_scorefar above the reference, orrejected_above_max> 0 → the scorer accepts something it should not. Read the offending program inRUN/r*/s*/prog.py, fix the evaluator or tightenscore_max, and start a new run: the archive is poisoned.- Every refine swarm in
rounds.jsonlparents the same broken program → a scorer hole made it the leader; same remedy. Diagnosed 2026-11-09: a hexagon run spent its last 40 minutes refining a program the scorer had wrongly rated 3.9e9. no_codeclimbing → the writer's reply has no fenced block definingentry. Check thatentryin the config matches the interface text and the initial program.duplicatesclose to the number of writer calls → the writer returns its parent unchanged; usually a writer that cannot see why the score is low. Add the useful metrics toshow_metrics, or raisewriter_effort.- Evaluation errors with
rc=125orrc=2→ the jail could not isolate (125) ornobodycannot enter the run directory (2). Put the run dir under/tmp/jail/<name>/and setchown_uid = 65534. - Spend per program well above a cent → the gateway's ledger shows the cost by role;
proposers were 30–39% of the spend in the 2026-11 ledgers, the writer the rest. Each
claude -pcall carries about 1,370 tokens of the CLI's own, so many tiny calls cost more than their prompts suggest. Fewer proposers (or N=0) and the writer's effort are the levers. pauses.jsonlappears and the run slows → the account's five-hour window passed 90% (non-Haiku calls) or 97% (Haiku). The gateway waits for the reset and leaves the wait out of the run clock; nothing to do unless it is urgent.- A run stops with
stopped_bystartingquota:→ seven-day utilisation reached 0.45. Stop launching runs this week, or raiseweek_stopingateway.pydeliberately.
Verification
python3 $S/test_evolve.py # offline: run, resume, status, score bound, sandbox prefix
python3 $S/evolve.py status RUN # programs > 1, spent <= cap + one round, best_score <= score_maxRe-score RUN/best_program.py with the evaluator yourself (in the sandbox) and compare it
with best_score. Programs that optimise with randomness or a time budget re-score lower;
two of three hexagon programs did on 2026-11-09. A bad success is a high score the
evaluator should not have given: check the best program against an independent
condition (a stricter geometric test, a known bound) before reporting a record.
Diagnosed failures
- 2026-11-09, hexagon packing. OpenEvolve's stock hexagon evaluator scored an outer
side of 1e-9 at 3.9e9: an absolute tolerance on an unnormalised cross product. Signal:
a score nine orders above the reference. Fixed by an area bound;
score_maxexists for this. - 2026-11-09, cost accounting.
claude -pwrote prompt cache at the 1-hour TTL (2x input) untilFORCE_PROMPT_CACHING_5M=1; the gateway sets it. Signal:cacheCreationEphemeral1hInputTokensnon-zero in the ledger. - 2026-11-09, effort.
MAX_THINKING_TOKENS=0andalwaysThinkingEnabled: falsedid not stop Haiku 5.5 from thinking, and an output cap below its need failed the call while billing it. Effort low is the floor; do not setmax_tokenstight. - 2026-11-09, usage window. Ten parallel runs used most of a five-hour window in about 70 minutes and the old gateway aborted runs at 90%. It now pauses and resumes instead.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

