cap-evolve

# cap-evolve

> A skills-native, host-agnostic harness for optimizing AI-agent capabilities
> (skills, tools/MCP, prompts) against your own evaluation, with honest
> train/val/test discipline enforced in code. The pipeline is a library of Agent
> Skills; the only shipped code is a tiny stdlib substrate.

If you are an agent driving cap-evolve: follow `RUN.md`. **Intake does the full
benchmark integration** — install the benchmark, implement the adapter
(`tasks · run_target` *or* a benchmark's own `run_batch` · `score`, plus optional
`trajectories(split)` returning the runner's native trace dir and a batched
`run_trials(tasks, ctx, *, n_trials, base_seed)` fast path; `materialize` is a pure
write and `live` a context manager, `apply` a back-compat hook), wire
`capability_sources` (data-model files the tools import), and author a
capability-scoped `optimizer/INSTRUCTIONS.md`; see `docs/ADAPTER_CONTRACT.md`. Pass
`cap-evolve check` (the hard gate), then run the phases intake → implement-and-check
→ baseline → <algorithm> → finalize → report. Each iteration the optimizer gets the
current best step's full trajectories, the selected capability skills, the per-task
impact of prior edits + the passing set to protect, and a STATE/MEMORY handover; it
may edit BOTH the prompt and the tools (incl. tool code). Never score the test split
except at finalize; gate acceptance on val with a paired significance gate.

## Start here
- [RUN.md](RUN.md): the top-level orchestration prompt (point any agent at this).
- [README.md](README.md): what it is, quickstart, comparison, skill library.
- [docs/ADAPTER_CONTRACT.md](docs/ADAPTER_CONTRACT.md): the adapter methods you implement.
- [docs/HONEST_EVAL.md](docs/HONEST_EVAL.md): the non-negotiable eval guarantees.

## Extend
- [docs/EXTENDING.md](docs/EXTENDING.md): add a capability/algorithm/optimizer skill.
- [CONTRIBUTING.md](CONTRIBUTING.md): dev setup and house rules.
- [docs/ROADMAP.md](docs/ROADMAP.md): prioritized next work.

## Skills (22)
Counting rule: one skill = one skills/<component>/<name>/SKILL.md, i.e. every row of
skills/_registry/manifest.json — phases + capabilities + algorithms + optimizers +
interventions + orchestrate.
- phases: intake, implement-and-check, baseline, evaluate, diagnose, gate, finalize, report
- capabilities: system-prompt, skill-package, tools, mcp-tool
- interventions: blackbox (how a candidate is DELIVERED to the model under test, vs what is edited)
- memory: wiki (which cross-iteration memory scheme, selected via `memory_skill` —
  md-files is the built-in default and needs no skill dir; wiki is the weakness-graph
  format extracted from evograph)
- algorithms (5): 3 run-executable + 2 agent-mode. Run-executable via `cap-evolve run`:
  hill-climb (--focus all|cyclic|hardest-first), gepa, skillopt (gepa & skillopt are the
  sample-efficient flagships). Agent-mode only (orchestration_mode: agent; their
  scripts/run.py is a loud guard, not a loop): agent-optimize and evograph
- optimizers: run-optimizer + optimizers/registry.yaml (claude-code, codex, gemini-cli,
  opencode, cursor, droid, copilot, kimi, pi, antigravity, openclaw, ibm-bob, generic,
  mock — one runner resolves the name; the agent set matches obra/superpowers)
- orchestrate: orchestrate, using-cap-evolve (session-start router)

## Examples
- [examples/toy_calc](examples/toy_calc): zero-API deterministic proof.
- [examples/tau2_airline](examples/tau2_airline): real tau2-bench with gpt-oss-120b.