autoresearch-create
Skill4 starsChanged 8 months agoMITView on GitHub
Set up and run an autonomous experiment loop for any optimization target. Gathers what to optimize, then starts the loop immediately. Use when asked to "run autoresearch", "optimize X in a loop", "set up autoresearch for X", or "start experiments".
2 other projects use a version of this, 2 of them edited.
Adopted by
Most recently taken up by
- davebcn87/pi-autoresearchedited, first seen Oct 8, 2026
- marcelopazzo/claude-code-autoresearchedited, first seen Apr 19, 2026
Other versions2 projects use a version of this.
Word for word · 0
None found.
Edited · 2
- davebcn87/pi-autoresearch★ 8.2k83% same
- marcelopazzo/claude-code-autoresearch★ 183% same
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository23 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- add-telegramSkill
- add-telegram-swarmSkill
- agent-browserSkill
- agent-harness-engineerSkill
- autoresearch-finalizeSkill
- fft-coder-opsSkill
- fft-dashboard-opsSkill
- fft-debugSkill
- fft-farm-bootstrapSkill
- fft-farm-onboardingSkill
- fft-farm-opsSkill
- fft-host-ipcSkill
- fft-setupSkill
- fft-telegram-opsSkill
- loopsSkill
- rapid-researchSkill
- skill-mechanicSkill
- web-searchSkill
- customizeSkill
- debugSkill
- setupSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
13 rules. About 0 are common, 6 are distinctive, and 7 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈6
Distinctive · estimate
This author's own choices. Worth a look.
7
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 18
Ask (or infer): Goal, Command, Metric (+ direction), Files in scope, Constraints.
- Distinctive: line 20
Read the source files. Understand the workload deeply before writing anything.
- Very specific: line 21
Write autoresearch.md and autoresearch.sh (see below).
- Very specific: line 55
Update autoresearch.md periodically — especially the "What's Been Tried" section — so resuming agents have full context.
- Distinctive: line 102
When this file exists:
- Very specific: line 104
If checks fail, run_experiment reports it clearly — log as checks_failed.
- Distinctive: line 109
When this file does not exist, everything behaves exactly as before — no changes to the loop.
- Distinctive: line 111
Keep output minimal: Only the last 80 lines of checks output are fed back to the agent on failure.
- Distinctive: line 123
LOOP FOREVER: Never ask "should I continue?" — the user expects autonomous work.
- Very specific: line 127
Watch the confidence score: After 3+ runs, log_experiment reports a confidence score (best improvement as a multiple of the session noise …
- Very specific: line 132
Resuming: if autoresearch.md exists, read it + git log, continue looping.
- Very specific: line 138
When you discover complex but promising optimizations that you won't pursue right now, append them as bullets to autoresearch.ideas.md.
- Very specific: line 144
If the user sends a message while an experiment is running, finish the current run_experiment + log_experiment cycle first, then …
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Autoresearch
Autonomous experiment loop: try ideas, keep what works, discard what doesn't, never stop.
Tools
init_experiment— configure session (name, metric, unit, direction). Call again to re-initialize with a new baseline when the optimization target changes.run_experiment— runs command, times it, captures output.log_experiment— records result.keepauto-commits.discard/crash/checks_failedauto-reverts code changes (autoresearch files preserved). Always include secondarymetricsdict. Dashboard: ctrl+x.
Setup
- Ask (or infer): Goal, Command, Metric (+ direction), Files in scope, Constraints.
git checkout -b autoresearch/<goal>-<date>- Read the source files. Understand the workload deeply before writing anything.
- Write
autoresearch.mdandautoresearch.sh(see below). Commit both. init_experiment→ run baseline →log_experiment→ start looping immediately.
autoresearch.md
This is the heart of the session. A fresh agent with no context should be able to read this file and run the loop effectively. Invest time making it excellent.
# Autoresearch: <goal>
## Objective
<Specific description of what we're optimizing and the workload.>
## Metrics
- **Primary**: <name> (<unit>, lower/higher is better) — the optimization target
- **Secondary**: <name>, <name>, ... — independent tradeoff monitors
## How to Run
`./autoresearch.sh` — outputs `METRIC name=number` lines.
## Files in Scope
<Every file the agent may modify, with a brief note on what it does.>
## Off Limits
<What must NOT be touched.>
## Constraints
<Hard rules: tests must pass, no new deps, etc.>
## What's Been Tried
<Update this section as experiments accumulate. Note key wins, dead ends,
and architectural insights so the agent doesn't repeat failed approaches.>Update autoresearch.md periodically — especially the "What's Been Tried" section — so resuming agents have full context.
autoresearch.sh
Bash script (set -euo pipefail) that: pre-checks fast (syntax errors in <1s), runs the benchmark, and outputs structured lines to stdout. Keep the script fast — every second is multiplied by hundreds of runs.
For fast, noisy benchmarks (< 5s), run the workload multiple times inside the script and report the median. This produces stable data points and makes the confidence score reliable from the start. Slow workloads (ML training, large builds) don't need this — single runs are fine.
Structured output
METRIC name=value— primary metric (must matchinit_experiment'smetric_name) and any secondary metrics. Parsed automatically byrun_experiment.
Design the script to inform optimization
The script should output whatever data helps you make better decisions in the next iteration. Think about what you'll need to see after each run to know where to focus:
- Phase timings when the workload has distinct stages
- Error counts, failure categories, or test names when checks can fail in different ways
- Memory usage, cache hit rates, or other runtime diagnostics when relevant
- Anything domain-specific that would help localize regressions or identify bottlenecks
The script runs the same code every iteration — but you can update it during the loop if you discover you need more signal. Add instrumentation as you learn what matters.
Agent-supplied ASI via log_experiment
Use log_experiment's asi parameter to annotate each run with whatever would help the next iteration make a better decision. Free-form key/value pairs — you decide what's worth recording. Don't repeat the description or raw output; capture what you'd lose after a context reset.
Annotate failures and crashes heavily. Discarded and crashed runs are reverted — the code changes are gone. The only record that survives is the description and ASI in autoresearch.jsonl. If you don't capture what you tried and why it failed, future iterations will waste time re-discovering the same dead ends.
autoresearch.config.json (optional)
JSON config file that lives in the pi session's working directory (ctx.cwd). Supported fields:
maxIterations(number) — maximum experiments before auto-stopping.workingDir(string) — override the directory for all autoresearch operations: file I/O (autoresearch.jsonl,autoresearch.md,autoresearch.sh,autoresearch.checks.sh,autoresearch.ideas.md), command execution, and git operations. Supports absolute paths or relative paths (resolved againstctx.cwd). The config file itself always stays inctx.cwd. Fails if the directory doesn't exist.
{
"workingDir": "/path/to/project",
"maxIterations": 50
}autoresearch.checks.sh (optional)
Bash script (set -euo pipefail) for backpressure/correctness checks: tests, types, lint, etc. Only create this file when the user's constraints require correctness validation (e.g., "tests must pass", "types must check").
When this file exists:
- Runs automatically after every passing benchmark in
run_experiment. - If checks fail,
run_experimentreports it clearly — log aschecks_failed. - Its execution time does NOT affect the primary metric.
- You cannot
keepa result when checks have failed. - Has a separate timeout (default 300s, configurable via
checks_timeout_seconds).
When this file does not exist, everything behaves exactly as before — no changes to the loop.
Keep output minimal. Only the last 80 lines of checks output are fed back to the agent on failure. Suppress verbose progress/success output and let only errors through. This keeps context lean and helps the agent pinpoint what broke.
#!/bin/bash
set -euo pipefail
# Example: run tests and typecheck — suppress success output, only show errors
pnpm test --run --reporter=dot 2>&1 | tail -50
pnpm typecheck 2>&1 | grep -i error || trueLoop Rules
LOOP FOREVER. Never ask "should I continue?" — the user expects autonomous work.
- Primary metric is king. Improved →
keep. Worse/equal →discard. Secondary metrics rarely affect this. - Annotate every run with
asi. Record what you learned — not what you did. What would help the next iteration or a fresh agent resuming this session? - Watch the confidence score. After 3+ runs,
log_experimentreports a confidence score (best improvement as a multiple of the session noise floor). ≥2.0× means the improvement is likely real. <1.0× means it's within noise — consider re-running to confirm before keeping. The score is advisory — it never auto-discards. - Simpler is better. Removing code for equal perf = keep. Ugly complexity for tiny gain = probably discard.
- Don't thrash. Repeatedly reverting the same idea? Try something structurally different.
- Crashes: fix if trivial, otherwise log and move on. Don't over-invest.
- Think longer when stuck. Re-read source files, study the profiling data, reason about what the CPU is actually doing. The best ideas come from deep understanding, not from trying random variations.
- Resuming: if
autoresearch.mdexists, read it + git log, continue looping.
NEVER STOP. The user may be away for hours. Keep going until interrupted.
Ideas Backlog
When you discover complex but promising optimizations that you won't pursue right now, append them as bullets to autoresearch.ideas.md. Don't let good ideas get lost.
On resume (context limit, crash), check autoresearch.ideas.md — prune stale/tried entries, experiment with the rest. When all paths are exhausted, delete the file and write a final summary.
User Messages During Experiments
If the user sends a message while an experiment is running, finish the current run_experiment + log_experiment cycle first, then incorporate their feedback in the next iteration. Don't abandon a running experiment.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

