docs — Working Notes & Autoresearch

This folder holds long-form research, design plans, and the DPAIA arena working log. Read this in addition to the root CLAUDE.md when changing files under docs/ or referencing autoresearch results.

The repo's four design tenets — small MCP tool surface; power lives in prompts and direct IntelliJ API usage; devrig is stateless; McpScriptContext is last-resort — are codified in PHILOSOPHY.md (and mirrored at runtime as mcp-steroid://skill/design-philosophy). Autoresearch and prompt- optimization work in this folder is the primary lever for those tenets: every measurement here ultimately feeds back into recipe quality, not new tools or new context methods.

For the test code that drives DPAIA / arena scenarios see test-experiments/CLAUDE.md.

Specs in this folder

Long-form contract documents owned by docs/ (read these directly rather than mirroring their contents into per-folder guides):

• PHILOSOPHY.md — the four design tenets.
• devrig-cli-contract.md — canonical devrig command grammar, generated-tool CLI, help/recovery, human and JSON output, open_project --wait, and agent-validation contract.
• devrig-naming.md — devrig CLI + stdio MCP project/backend naming contract (slug rule, bootHash, archiveSha256, actions[].argv, on-demand routing).
• devrig-scanning-research.md — decision record for on-demand rebuildSnapshot() vs background scanners (option A).
• install-scripts-contract.md — locked contract (issue #398): the bootstrap installers install devrig and register it on PATH, NOTHING else — never auto-register devrig with agents, never auto-install the IDE plugin; those commands are only promoted to the user in the devrig install devrig info message.
• devrig-deployment-spec.md — historical v7 design record for ~/.mcp-steroid/ layout, wrapper ownership, the bundled runtime, cache/GC ideas, and a native-binary alternative. Its wizard and upgrade proposals are superseded; current install behavior lives in install-scripts-contract.md, current commands in devrig-cli-contract.md, and current updates in updates-check/devrig-auto-update.md.
• devrig-remote-development-backend-e2e.md — shipped IU-262 native Remote Development backend contract, Docker fixture, lifecycle evidence, and follow-up risks.
• headless-agent-guidance.md — task-only Claude/Codex autonomy follow-up: clean-machine backend discovery, frontendless readiness, Maven import, hierarchy scoring, credential isolation, and measured agent evidence.
• native-mcp-tools-design.md — research record + spec for listing/calling the IntelliJ MCP Server plugin's native tools: validated LIST/CALL recipes (261→master), the devrig native-tools bridge endpoint + the historical devrig project tools CLI proposal (the active TODO now requires a new top-level route because projects/project alias the generated list_projects leaf), and the mcp-steroid://skill/native-mcp-tools index + dynamic per-tool resource pages.
• ARCHITECTURE.md — top-level architecture map.
• TESTING-STRATEGY.md — test layout + per-module scoping rules.

Prompt optimization (autoresearch)

MCP Steroid serves prompt resources (mcp-steroid:// URIs) that guide AI agents. Optimizing these prompts is an iterative process — running curated DPAIA tasks under controlled conditions, measuring tool calls and tokens, then narrowing the gap between agent behavior and expected behavior with prompt-only edits.

Findings, comparison tables, and the Karpathy-style optimization loop prompts:

• docs/autoresearch-findings.md — high-level summary
• docs/autoresearch/ — loop prompts
• docs/arena-3pass-results.md — full 3-pass comparison table

Key findings (from 51 arena runs + 2 autoresearch cycles)

• MCP server instructions (prompts/src/main/prompts/mcp-steroid-info.md) are metadata context, not behavioral directives — agents don't follow them during planning.
• Tool descriptions are schema reference — MANDATORY warnings in them don't change behavior.
• MCP resources (84 available via ReadMcpResourceTool) are rarely read (0/69 runs in baseline).
• Arena prompt recipes DO work — agents follow first-call exec_code recipes verbatim.
• exec_code output drives next-step decisions — agents act on compile results immediately.

Where agent information lives (priority order)

1. User prompt (arena task) — agents follow this. Put recipes here.
2. exec_code output — agents act on results. Put suggestions here.
3. System prompt (MCP server instructions) — background context only.
4. Tool schema description — reference material, not directives.
5. MCP resources — rarely accessed unless explicitly fetched.

Prompt files

Active DPAIA working notes

• Repo-root ../TASKS.md is the active task list (open tasks only). The former repo-root MEMORY.md DPAIA handoff log was distilled into the docs below + git log and removed — durable findings now live in autoresearch-findings.md / arena-3pass-results.md, not a running scratch file.
• New DPAIA ideas must also be logged in ../TODO-DPAIA.md (or sibling).
• Direction changes require 3 run-agent.sh reviews and consensus before selecting the next low-hanging fruit.
• Constraints for the autoresearch track: do not add McpSteroid* interface methods; do not add new MCP tools. Improvements must be prompt-only (skill articles, tool descriptions, system prompt text).

History (2026-04-26 → 2026-04-27)

Full chronological run-by-run history is intentionally NOT mirrored here — it churned weekly during active iteration and lives in:

• arena-3pass-results.md — measured tables.
• autoresearch-findings.md — synthesized takeaways.
• git log per file — diff per change (see anchors below). The former repo-root MEMORY.md running log (which once held the day-to-day "recent facts" + next-step consensus) was distilled into these docs and removed; recover any specific run note from git log -- MEMORY.md.

When picking up the autoresearch track:

1. Read autoresearch-findings.md (+ git log -- MEMORY.md for the last next-step consensus, if needed).
2. Confirm the consensus is still load-bearing by checking git log since that note was written.
3. Run the most recent baseline scenario before changing prompts so you have a fresh comparison point.

Latest landmark anchors (with git log commands for digging)

# Gradle abort root-cause fix — ProjectDataImportListener.onFinalTasksFinished as the sync boundary;
# DPAIA Microshop on JDK 24 because Gradle 8.14.3 rejects Java 25 daemon.
git log --oneline -- test-integration/src/main/kotlin/com/jonnyzzz/mcpSteroid/integration/infra/intelliJ.kt
git log --oneline --grep="onFinalTasksFinished\|JDK 24\|Gradle abort"

# Gradle IDE guidance — arena prompts inline ProjectTaskManager.build(*modules).await() before Bash fallback.
git log --oneline -- prompts/src/main/prompts/skill/execute-code-gradle.md
git log --oneline -- test-experiments/src/test/kotlin/com/jonnyzzz/mcpSteroid/integration/arena/ArenaTestRunner.kt

# Apply-patch history — the steroid_apply_patch tool (removed May 2026) and the applyPatch { } DSL (removed July 2026, #206).
git log --oneline --grep="apply.patch\|ApplyPatch"

# Aborted-build result-boundary guidance — ExecuteCodeToolHandler appends REQUIRED ACTION hint
# (uses ExecuteCodeGradlePromptArticle().uri / ExecuteCodeMavenPromptArticle().uri).
git log --oneline -- ij-plugin/src/main/kotlin/com/jonnyzzz/mcpSteroid/server/ExecuteCodeToolHandler.kt
git log --oneline --grep="aborted=true\|REQUIRED ACTION\|build-abort\|BuildAbortGuidance"

# IntelliJ monorepo lookup — prefer Observation.awaitConfiguration(project) + smartReadAction(project)
# over waitForSmartMode() for indexed reads (regression: IntelliJThisLoggerLookupTest).
git log --oneline --grep="awaitConfiguration\|smartReadAction\|ThisLoggerLookup"