autoresearch
AGENTS.md571 starsChanged 7 months agoMITView on GitHub
This file provides guidance to Codex (Codex.ai/code) when working with code in this repository. autoresearch is a fully automated software development tool: given a GitHub Issue number, it implements the feature through multi-agent iterative review, then auto-creates a PR, merges it, and closes the Issue. The entire runtime is a single Bash script (run.sh). There are no build/test/lint commands for autoresearch itself. Shell-based regression tests live under tests/, including: - tests/test_extract_score.sh - tests/test_agent_logic.sh - tests/test_archive.sh - tests/test_context_overflow.sh - tests/test_cleanup.sh Before…
Where files are headingNew AGENTS.md files have more rules about keeping secrets out of the code, and fewer about code style.
Of the AGENTS.md files we read, the 32,671 first seen in the last six months, against the older ones: the kind of rule that gained most share, and the kind that lost most. About files like this in general, not this one, and an estimate, since a kind is a pattern in the words.
In this repository5 other files it gives its agents.
- agents/claude.mdCLAUDE.md
- CLAUDE.mdCLAUDE.md
- lib/CLAUDE.mdCLAUDE.md
- tests/CLAUDE.mdCLAUDE.md
- prdSkill
The file
We split this AGENTS.md into the rules it gives an agent
10 rules. About 0 are common, 1 is distinctive, and 9 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
9
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 34
Before a new run starts (and not in continue mode), run.sh automatically archives old workflow data:
- Very specific: line 37
If the target archive directory already exists, it appends a numeric suffix (e.g., -1, -2).
- Very specific: line 50
If planning fails (no tasks.json generated), falls back to original mode: implement entire Issue in one go.
- Very specific: line 79
No tasks.json (planning failure) → backward-compatible original mode
- Very specific: line 80
When reviewing archived workflow iterations, treat .autoresearch/workflows/issue-N/planning.log as the source of truth for the original …
- Very specific: line 116
When merge/cleanup operations are non-fatal by design, tests should mock gh/git failures and assert both log.md side effects and that later …
- Very specific: line 123
Default passing score is 85/100, configurable via PASSING_SCORE env var
- Distinctive: line 145
If browser tools unavailable: UI verification is disabled, continues with code review only
- Very specific: line 146
If screenshot fails: Logs warning, returns pass: true with degraded feedback
- Very specific: line 147
If LLM verification fails: Retries up to 3 times, then returns pass: true with warning
≈ “Common” is an estimate: at least 1 in 5 of the 41,581 AGENTS.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
AGENTS.md
This file provides guidance to Codex (Codex.ai/code) when working with code in this repository.
Project Overview
autoresearch is a fully automated software development tool: given a GitHub Issue number, it implements the feature through multi-agent iterative review, then auto-creates a PR, merges it, and closes the Issue. The entire runtime is a single Bash script (run.sh).
Running
# Process Issue #N in the current directory's project
./run.sh 42
# With project path, max iterations, or continue mode
./run.sh -p /path/to/project 42 16
./run.sh -c 42 10 # Continue from last interrupted iteration
# Skip archiving of previous workflows (useful for debugging)
./run.sh --no-archive 42There are no build/test/lint commands for autoresearch itself. Shell-based regression tests live under tests/, including:
tests/test_extract_score.shtests/test_agent_logic.shtests/test_archive.shtests/test_context_overflow.shtests/test_cleanup.sh
Architecture
Archiving Mechanism
Before a new run starts (and not in continue mode), run.sh automatically archives old workflow data:
- Scans
.autoresearch/workflows/forissue-*directories. - Moves non-current issue directories to
.autoresearch/archive/YYYY-MM-DD-issue-N/. - If the target archive directory already exists, it appends a numeric suffix (e.g.,
-1,-2).
Iteration Loop
Planning Phase: First agent reads Issue → outputs tasks.json with subtask breakdown
Iteration 1: Agent implements current subtask → tests run
Iteration 2+: Agent round-robin review + fix per subtask
→ Score >= 85? → mark subtask passed → advance to next subtask
→ Score < 85? → agent fixes based on review feedback → next iteration
→ All subtasks passed? → auto commit/PR/merge/close IssueIf planning fails (no tasks.json generated), falls back to original mode: implement entire Issue in one go.
Agent rotation (from iteration 2): (iter - 1) % N where N = number of agents.
Global Cleanup / Traps
run.shinstalls globalEXIT,INT,TERM, andDEBUGtraps near the spinner helpers.cleanup()must stay idempotent because interrupt paths can otherwise re-enter during exit handling.- Signal traps should disable the
EXITtrap before exiting, otherwise abnormal exits will append duplicate interruption records tolog.md. - Child-process cleanup is recursive via
collect_descendant_pids()and should keep apsfallback becausepgrep -Pis not reliable in every macOS environment. DEBUGtrap records the last executed command inLAST_COMMANDto help identify which command triggeredset -eexits.cleanup()logs exit code, exit reason, and the last executed command to bothlog.mdand console for debugging unexpected exits.
Subtask System (tasks.json)
Planning phase (before iteration 1) asks the first agent to split the Issue into subtasks, saved to .autoresearch/workflows/issue-N/tasks.json:
{
"issueNumber": 42,
"subtasks": [
{ "id": "T-001", "title": "...", "description": "...", "acceptanceCriteria": [...], "priority": 1, "passes": false }
]
}- Each iteration focuses on one
passes: falsesubtask - Quality gate passes →
mark_subtask_passed()setspasses: true all_subtasks_passed()→ enters PR/merge flow- No tasks.json (planning failure) → backward-compatible original mode
- When reviewing archived workflow iterations, treat
.autoresearch/workflows/issue-N/planning.logas the source of truth for the original subtask acceptance criteria.tasks.jsoninside the workflow can contain placeholder or truncated subtask text in later iterations and is not always reliable for audit scoring.
Agent Invocation
Agents are external CLIs called via run_with_retry() with exponential backoff:
- Codex:
Codex -p "$prompt" --dangerously-skip-permissions - Codex:
codex exec --full-auto "$prompt" - OpenCode:
opencode run "$prompt"
Prompt Assembly
Each agent prompt composes these parts:
- Task context (Issue info, project path, language, iteration)
- Subtask section (current subtask details + progress) — from
get_subtask_section()/get_subtask_review_section() program.mdcontent — global rules and constraints- Agent-specific instructions from
agents/<name>.md
For review prompts, only task context + subtask review section + agent instructions are included (no program.md).
Configuration Override (Two-Tier)
Project-level configs in $PROJECT_ROOT/.autoresearch/ take precedence over defaults in the autoresearch directory:
agents/<name>.md— agent persona and review rubricprogram.md— implementation rules and code standards
Score Extraction
extract_score() parses free-text review output using 6 cascading regex patterns (X/100, 评分: X/100, 总分 table, X/10, 评分: X, 评分: X). If no score is found, defaults to 0; review functions fall back to 50 on extraction failure.
Continue Mode (-c)
Restores state from .autoresearch/workflows/issue-N/ log files: iteration count, last score, consecutive failure count, last review feedback, and subtask state (from tasks.json). MAX_ITERATIONS becomes last_iteration + new_count.
Git / GitHub Failure Paths
- The final PR flow in
run.shis stillset -esensitive;gh pr mergemust be wrapped with explicit failure handling rather than called bare. - When merge/cleanup operations are non-fatal by design, tests should mock
gh/gitfailures and assert bothlog.mdside effects and that later cleanup steps still execute.
Key Design Constraints
- Agents cannot push to remote, close Issues, create PRs, or modify CI/CD — only
run.shperforms those privileged operations after quality gates pass program.mdcontains code standards for Go/Python/TypeScript/Rust/Frontend — the README advises trimming to only the target project's language to save tokens- Consecutive iteration failures >= 2 triggers a hard stop
- Default passing score is 85/100, configurable via
PASSING_SCOREenv var
UI Verification
The UI verification system runs browser-based validation for UI-type subtasks ("type": "ui").
Configuration
Environment variables (can also be set in shell):
UI_VERIFY_ENABLED: Enable/disable UI verification (default:yes)UI_VERIFY_TIMEOUT: Dev server wait timeout in seconds (default:60)UI_DEV_PORT: Dev server port (default: auto-detect or3000)
Process
- Dependency Check: During
check_dependencies(), detects if playwright or chrome-devtools MCP is available - Dev Server Detection: Automatically detects dev server command based on project type
- Screenshot Capture: Uses multiple fallback tools (playwright → npx playwright → google-chrome → chromium → chromium-browser)
- LLM Verification: Sends screenshot to LLM for validation against UI standards
Fallback Behavior
- If browser tools unavailable: UI verification is disabled, continues with code review only
- If screenshot fails: Logs warning, returns
pass: truewith degraded feedback - If LLM verification fails: Retries up to 3 times, then returns
pass: truewith warning
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

