auto-paper-improvement-loop
Skill9 starsChanged 3 months agoMITView on GitHub
Prompt-only by default in the lightweight pack; Claude/Gemini reviewer transport is explicit opt-in. Autonomously improve a generated paper via Gemini review through gemini-review MCP → implement fixes → recompile, for 2 rounds. Use when user says \"改论文\", \"improve paper\", \"论文润色循环\", \"auto improve\", or wants to iteratively polish a generated paper.
4 other projects use a version of this, 4 of them edited.
Adopted by
Most recently taken up by
- talhaMah56/claude_skillsedited, first seen Sep 12, 2026
- wanshuiyin/Auto-claude-code-research-in-sleepedited, first seen Jul 15, 2026
- raja21068/AutoResearchedited, first seen May 5, 2026
- Yusong-Enceladus/claude-skillsedited, first seen Mar 18, 2026
Other versions4 projects use a version of this.
Word for word · 0
None found.
Edited · 4
- wanshuiyin/Auto-claude-code-research-in-sleep★ 19k16% same
- Yusong-Enceladus/claude-skills★ 889% same
- raja21068/AutoResearch★ 223% same
- talhaMah56/claude_skills★ 015% same
Wrote their own version · 0
None found.
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository47 other files it gives its agents.
- autodl-hpcSkill
- experiment-bridgeSkill
- experiment-planSkill
- human-toneSkill
- idea-discoverySkill
- paper-visualizationSkill
- paper-writingSkill
- rebuttalSkill
- research-blueprintSkill
- research-pipelineSkill
- research-repo-architectSkill
- research-reviewSkill
- resubmit-pipelineSkill
- autodl-hpcSkill
- auto-paper-improvement-loopSkill
- auto-review-loopSkill
- novelty-checkSkill
- paper-figureSkill
- paper-planSkill
- paper-writeSkill
- research-refineSkill
- research-reviewSkill
- experiment-bridgeSkill
- experiment-planSkill
- auto-review-loopSkill
- grant-proposalSkill
- idea-creatorSkill
- idea-discoverySkill
- novelty-checkSkill
- paper-figureSkill
- paper-planSkill
- paper-posterSkill
- paper-slidesSkill
- paper-writeSkill
- paper-writingSkill
- research-refineSkill
- research-reviewSkill
- human-toneSkill
- idea-discoverySkill
- paper-visualizationSkill
- paper-writingSkill
- rebuttalSkill
- research-blueprintSkill
- research-pipelineSkill
- research-repo-architectSkill
- research-reviewSkill
- resubmit-pipelineSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
16 rules. 0 are common, 6 are distinctive, and 10 are very specific to this project.
0
Common
Most of the 4 edited versions we read keep these.
6
Distinctive
This author's own choices. Worth a look.
10
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 14
Write the review prompt under review-prompts/ and ask the user to run it in a separate conversation with the relevant skills enabled.
- Distinctive: line 16
Do not call Claude/Gemini reviewer MCP bridges, APIs, or follow-up status tools by default.
- Very specific: line 18
Use pasted review text as the external verdict, then write compact summaries such as REVIEW_SUMMARY.md, NEXT_ACTIONS.md, or the …
- Distinctive: line 20
Keep heavy experiments AutoDL/HPC-gated and user-approved.
- Very specific: line 49
If the context window fills up mid-loop, Codex auto-compacts.
- Very specific: line 61
On startup: if PAPER_IMPROVEMENT_STATE.json exists with "status": "in_progress" AND timestamp is within 24 hours, read it + …
- Very specific: line 111
After this start call, immediately save the returned jobId and poll mcp__gemini-review__review_status with a bounded waitSeconds until …
- Distinctive: line 163
Verify: 0 undefined references, 0 undefined citations.
- Very specific: line 167
Use mcp__gemini-review__review_reply_start with the saved completed threadId:
- Distinctive: line 193
Add controlled synthetic experiments validating theory
- Distinctive: line 207
After the final recompilation, run a format compliance check:
- Distinctive: line 237
If any overfull hbox > 10pt is found, fix it and recompile before documenting.
- Very specific: line 241
Create PAPER_IMPROVEMENT_LOG.md in the paper directory:
- Very specific: line 298
After each round's review AND at final completion, check ~/.codex/feishu.json:
- Very specific: line 301
If config absent or mode "off": skip entirely (no-op)
- Very specific: line 323
Large file handling: If the Write tool fails due to file size, immediately retry using Bash (cat << 'EOF' > file) to write in chunks.
“Common” is measured: kept by at least 6 in 10 of the 4 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Override for Codex users who want Gemini, not a second Codex agent, to act as the reviewer. Install this package after
skills/skills-codex/*.
Customized Pack Defaults
This reviewer overlay is opt-in transport only. In the customized lightweight pack, default to prompt-only review unless the user explicitly asks to use this overlay reviewer backend:
- Write the review prompt under
review-prompts/and ask the user to run it in a separate conversation with the relevant skills enabled. - Do not call Claude/Gemini reviewer MCP bridges, APIs, or follow-up status tools by default.
- Use pasted review text as the external verdict, then write compact summaries
such as
REVIEW_SUMMARY.md,NEXT_ACTIONS.md, or the skill-specific log. - Keep heavy experiments AutoDL/HPC-gated and user-approved; local work stays at edits, tests, lint, dry-runs, and tiny smoke checks.
Auto Paper Improvement Loop: Review → Fix → Recompile
Autonomously improve the paper at: $ARGUMENTS
Context
This skill is designed to run after Workflow 3 (/paper-plan → /paper-figure → /paper-write → /paper-compile). It takes a compiled paper and iteratively improves it through external LLM review.
Unlike /auto-review-loop (which iterates on research — running experiments, collecting data, rewriting narrative), this skill iterates on paper writing quality — fixing theoretical inconsistencies, softening overclaims, adding missing content, and improving presentation.
Constants
- MAX_ROUNDS = 2 — Two rounds of review→fix→recompile. Empirically, Round 1 catches structural issues (4→6/10), Round 2 catches remaining presentation issues (6→7/10). Diminishing returns beyond 2 rounds for writing-only improvements.
- REVIEWER_MODEL =
gemini-review— Gemini reviewer invoked through the localgemini-reviewMCP bridge. SetGEMINI_REVIEW_MODELif you need a specific Gemini model override. - REVIEW_LOG =
PAPER_IMPROVEMENT_LOG.md— Cumulative log of all rounds, stored in paper directory. - HUMAN_CHECKPOINT = false — When
true, pause after each round's review and present score + weaknesses to the user. The user can approve fixes, provide custom modification instructions, skip specific fixes, or stop early. Whenfalse(default), runs fully autonomously.
💡 Override:
/auto-paper-improvement-loop "paper/" — human checkpoint: true
Inputs
- Compiled paper —
paper/main.pdf+ LaTeX source files - All section
.texfiles — concatenated for review prompt
State Persistence (Compact Recovery)
If the context window fills up mid-loop, Codex auto-compacts. To recover, this skill writes PAPER_IMPROVEMENT_STATE.json after each round:
{
"current_round": 1,
"thread_id": "019ce736-...",
"last_score": 6,
"status": "in_progress",
"timestamp": "2026-03-13T21:00:00"
}On startup: if PAPER_IMPROVEMENT_STATE.json exists with "status": "in_progress" AND timestamp is within 24 hours, read it + PAPER_IMPROVEMENT_LOG.md to recover context, then resume from the next round. Otherwise (file absent, "status": "completed", or older than 24 hours), start fresh.
After each round: overwrite the state file. On completion: set "status": "completed".
Workflow
Step 0: Preserve Original
cp paper/main.pdf paper/main_round0_original.pdfStep 1: Collect Paper Text
Concatenate all section files into a single text block for the review prompt:
# Collect all sections in order
for f in paper/sections/*.tex; do
echo "% === $(basename $f) ==="
cat "$f"
done > /tmp/paper_full_text.txtStep 2: Round 1 Review
Send the full paper text to Gemini review:
mcp__gemini-review__review_start:
prompt: |
You are reviewing a [VENUE] paper. Please provide a detailed, structured review.
## Full Paper Text:
[paste concatenated sections]
## Review Instructions
Please act as a senior ML reviewer ([VENUE] level). Provide:
1. **Overall Score** (1-10, where 6 = weak accept, 7 = accept)
2. **Summary** (2-3 sentences)
3. **Strengths** (bullet list, ranked)
4. **Weaknesses** (bullet list, ranked: CRITICAL > MAJOR > MINOR)
5. **For each CRITICAL/MAJOR weakness**: A specific, actionable fix
6. **Missing References** (if any)
7. **Verdict**: Ready for submission? Yes / Almost / No
Focus on: theoretical rigor, claims vs evidence alignment, writing clarity,
self-containedness, notation consistency.After this start call, immediately save the returned jobId and poll mcp__gemini-review__review_status with a bounded waitSeconds until done=true. Treat the completed status payload's response as the reviewer output, and save the completed threadId for any follow-up round.
Save the returned jobId, poll mcp__gemini-review__review_status until done=true, then save the completed threadId for Round 2.
Step 2b: Human Checkpoint (if enabled)
Skip if HUMAN_CHECKPOINT = false.
Present the review results and wait for user input:
📋 Round 1 review complete.
Score: X/10 — [verdict]
Key weaknesses (by severity):
1. [CRITICAL] ...
2. [MAJOR] ...
3. [MINOR] ...
Reply "go" to implement all fixes, give custom instructions, "skip 2" to skip specific fixes, or "stop" to end.Parse user response same as /auto-review-loop: approve / custom instructions / skip / stop.
Step 3: Implement Round 1 Fixes
Parse the review and implement fixes by severity:
Priority order:
- CRITICAL fixes (assumption mismatches, internal contradictions)
- MAJOR fixes (overclaims, missing content, notation issues)
- MINOR fixes (if time permits)
Common fix patterns:
| Issue | Fix Pattern |
| Assumption-model mismatch | Rewrite assumption to match the model, add formal proposition bridging the gap |
| Overclaims | Soften language: "validate" → "demonstrate practical relevance", "comparable" → "qualitatively competitive" |
| Missing metrics | Add quantitative table with honest parameter counts and caveats |
| Theorem not self-contained | Add "Interpretation" paragraph listing all dependencies |
| Notation confusion | Rename conflicting symbols globally, add Notation paragraph |
| Missing references | Add to references.bib, cite in appropriate locations |
| Theory-practice gap | Explicitly frame theory as idealized; add synthetic validation subsection |
Step 4: Recompile Round 1
cd paper && latexmk -C && latexmk -pdf -interaction=nonstopmode -halt-on-error main.tex
cp main.pdf main_round1.pdfVerify: 0 undefined references, 0 undefined citations.
Step 5: Round 2 Review
Use mcp__gemini-review__review_reply_start with the saved completed threadId:
mcp__gemini-review__review_reply_start:
threadId: [saved from Round 1]
prompt: |
[Round 2 update]
Since your last review, we have implemented:
1. [Fix 1]: [description]
2. [Fix 2]: [description]
...
Please re-score and re-assess. Same format:
Score, Summary, Strengths, Weaknesses, Actionable fixes, Verdict.After this start call, immediately save the returned jobId and poll mcp__gemini-review__review_status with a bounded waitSeconds until done=true. Treat the completed status payload's response as the reviewer output, and save the completed threadId for any follow-up round.
Step 5b: Human Checkpoint (if enabled)
Skip if HUMAN_CHECKPOINT = false. Same as Step 2b — present Round 2 review, wait for user input.
Step 6: Implement Round 2 Fixes
Same process as Step 3. Typical Round 2 fixes:
- Add controlled synthetic experiments validating theory
- Further soften any remaining overclaims
- Formalize informal arguments (e.g., truncation → formal proposition)
- Strengthen limitations section
Step 7: Recompile Round 2
cd paper && latexmk -C && latexmk -pdf -interaction=nonstopmode -halt-on-error main.tex
cp main.pdf main_round2.pdfStep 8: Format Check
After the final recompilation, run a format compliance check:
# 1. Page count vs venue limit
PAGES=$(pdfinfo paper/main.pdf | grep Pages | awk '{print $2}')
echo "Pages: $PAGES (limit: 9 main body for ICLR/NeurIPS)"
# 2. Overfull hbox warnings (content exceeding margins)
OVERFULL=$(grep -c "Overfull" paper/main.log 2>/dev/null || echo 0)
echo "Overfull hbox warnings: $OVERFULL"
grep "Overfull" paper/main.log 2>/dev/null | head -10
# 3. Underfull hbox warnings (loose spacing)
UNDERFULL=$(grep -c "Underfull" paper/main.log 2>/dev/null || echo 0)
echo "Underfull hbox warnings: $UNDERFULL"
# 4. Bad boxes summary
grep -c "badness" paper/main.log 2>/dev/null || echo "0 badness warnings"Auto-fix patterns:
| Issue | Fix |
| Overfull hbox in equation | Wrap in \resizebox or split with \split/aligned |
| Overfull hbox in table | Reduce font (\small/\footnotesize) or use \resizebox{\linewidth}{!}{...} |
| Overfull hbox in text | Rephrase sentence or add \allowbreak / \- hints |
| Over page limit | Move content to appendix, compress tables, reduce figure sizes |
| Underfull hbox (loose) | Rephrase for better line filling or add \looseness=-1 |
If any overfull hbox > 10pt is found, fix it and recompile before documenting.
Step 9: Document Results
Create PAPER_IMPROVEMENT_LOG.md in the paper directory:
# Paper Improvement Log
## Score Progression
| Round | Score | Verdict | Key Changes |
|-------|-------|---------|-------------|
| Round 0 (original) | X/10 | No/Almost/Yes | Baseline |
| Round 1 | Y/10 | No/Almost/Yes | [summary of fixes] |
| Round 2 | Z/10 | No/Almost/Yes | [summary of fixes] |
## Round 1 Review & Fixes
<details>
<summary>Gemini Review (Round 1)</summary>
[Full raw review text, verbatim]
</details>
### Fixes Implemented
1. [Fix description]
2. [Fix description]
...
## Round 2 Review & Fixes
<details>
<summary>Gemini Review (Round 2)</summary>
[Full raw review text, verbatim]
</details>
### Fixes Implemented
1. [Fix description]
2. [Fix description]
...
## PDFs
- `main_round0_original.pdf` — Original generated paper
- `main_round1.pdf` — After Round 1 fixes
- `main_round2.pdf` — Final version after Round 2 fixesStep 9: Summary
Report to user:
- Score progression table
- Number of CRITICAL/MAJOR/MINOR issues fixed per round
- Final page count
- Remaining issues (if any)
Feishu Notification (if configured)
After each round's review AND at final completion, check ~/.codex/feishu.json:
- After each round: Send
review_scored— "Round N: X/10 — [key changes]" - After final round: Send
pipeline_done— score progression table + final page count - If config absent or mode
"off": skip entirely (no-op)
Output
paper/
├── main_round0_original.pdf # Original
├── main_round1.pdf # After Round 1
├── main_round2.pdf # After Round 2 (final)
├── main.pdf # = main_round2.pdf
└── PAPER_IMPROVEMENT_LOG.md # Full review log with scoresOutput Protocols
Follow these shared protocols for all output files:
- Output Versioning Protocol — write timestamped file first, then copy to fixed name
- Output Manifest Protocol — log every output to docs/project/OUTPUT_MANIFEST.md
- Output Language Protocol — respect the project's language setting
Key Rules
- Large file handling: If the Write tool fails due to file size, immediately retry using Bash (
cat << 'EOF' > file) to write in chunks. Do NOT ask the user for permission — just do it silently. - Preserve all PDF versions — user needs to compare progression
- Save FULL raw review text — do not summarize or truncate Gemini reviewer responses
- Use
mcp__gemini-review__review_reply_startplusmcp__gemini-review__review_statusfor Round 2 to maintain conversation context - Always recompile after fixes — verify 0 errors before proceeding
- Do not fabricate experimental results — synthetic validation must describe methodology, not invent numbers
- Respect the paper's claims — soften overclaims rather than adding unsupported new claims
- Global consistency — when renaming notation or softening claims, check ALL files (abstract, intro, method, experiments, theory sections, conclusion, tables, figure captions)
Typical Score Progression
Based on end-to-end testing on a real theory-paper run:
| Round | Score | Key Improvements |
| Round 0 | 4/10 (content) | Baseline: assumption-model mismatch, overclaims, notation issues |
| Round 1 | 6/10 (content) | Fixed assumptions, softened claims, added interpretation, renamed notation |
| Round 2 | 7/10 (content) | Added synthetic validation, formal truncation proposition, stronger limitations |
| Round 3 | 5→8.5/10 (format) | Removed hero fig, appendix, compressed conclusion, fixed overfull hbox |
+4.5 points across 3 rounds (2 content + 1 format) is typical for a well-structured but rough first draft. Final state at submission: clean overfull-hbox count and venue-format-compliant length.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

