research-refine
Skill16 starsChanged 6 months agoMITView on GitHub
Turn a vague research direction into a problem-anchored, elegant, frontier-aware, implementation-oriented method plan via iterative GPT-5.4 review. Use when the user says \"refine my approach\", \"\u5e2e\u6211\u7ec6\u5316\u65b9\u6848\", \"decompose this problem\", \"\u6253\u78e8idea\", \"refine research plan\", \"\u7ec6\u5316\u7814\u7a76\u65b9\u6848\", or wants a concrete research method that stays simple, focused, and top-venue ready instead of a vague or overbuilt idea.
8 other projects use a version of this, 7 of them edited.
Adopted by
Most recently taken up by
- Yulivu/debuffer-skillsedited, first seen Oct 13, 2026
- wangyi0403/dl-research-workflowtheir own version, first seen Oct 13, 2026
- yvesory/academic-research-suiteedited, first seen Oct 9, 2026
- talhaMah56/claude_skillsedited, first seen Sep 12, 2026
- wanshuiyin/Auto-claude-code-research-in-sleepedited, first seen Jul 15, 2026
- zjunlp/Mechanistedited, first seen Jul 11, 2026
Other versions8 projects use a version of this.
Word for word · 0
None found.
Edited · 7
- wanshuiyin/Auto-claude-code-research-in-sleep★ 19k75% same
- zjunlp/Mechanist★ 7583% same
- Chanw-research/claude-code-paper-writing★ 4286% same
- Yulivu/debuffer-skills★ 985% same
- yvesory/academic-research-suite★ 980% same
- raja21068/AutoResearch★ 286% same
- talhaMah56/claude_skills★ 075% same
Wrote their own version · 1
- wangyi0403/dl-research-workflow★ 4same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository56 other files it gives its agents.
- argument-stress-testSkill
- citation-auditSkill
- comm-lit-review-claude-singleSkill
- comparative-lawSkill
- exa-searchSkill
- feishu-notifySkill
- figure-descriptionSkill
- figure-specSkill
- formula-derivationSkill
- grant-proposalSkill
- legal-writingSkill
- llm-wikiSkill
- mermaid-diagramSkill
- novelty-checkSkill
- overleaf-syncSkill
- paper-claim-auditSkill
- paper-compileSkill
- paper-figureSkill
- paper-illustrationSkill
- paper-planSkill
- paper-posterSkill
- paper-slidesSkill
- paper-writeSkill
- paper-writingSkill
- prism-pipelineSkill
- proof-checkerSkill
- proof-writerSkill
- rebuttalSkill
- research-litSkill
- research-pipelineSkill
- research-refine-pipelineSkill
- research-refineSkill
- research-reviewSkill
- research-wikiSkill
- semantic-scholarSkill
- comm-lit-reviewSkill
- feishu-notifySkill
- formula-derivationSkill
- grant-proposalSkill
- mermaid-diagramSkill
- novelty-checkSkill
- paper-compileSkill
- paper-figureSkill
- paper-illustrationSkill
- paper-planSkill
- paper-posterSkill
- paper-slidesSkill
- paper-writeSkill
- paper-writingSkill
- proof-writerSkill
- rebuttalSkill
- research-litSkill
- research-pipelineSkill
- research-refine-pipelineSkill
- research-reviewSkill
- statute-case-mapperSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
42 rules. 16 are common, 14 are distinctive, and 12 are very specific to this project.
16
Common
Most of the 8 edited versions we read keep these.
14
Distinctive
This author's own choices. Worth a look.
12
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Common: line 17
The smallest adequate mechanism wins: Prefer the minimal intervention that directly fixes the bottleneck.
- Common: line 18
One paper, one dominant contribution: Prefer one sharp thesis plus at most one supporting contribution.
- Common: line 19
Modern leverage is a prior, not a decoration: When LLM / VLM / Diffusion / RL / distillation / inference-time scaling naturally fit the …
- Very specific: line 62
Write after each completed phase.
- Very specific: line 81
Every round-N-refinement.md must contain a full anchored proposal, not just incremental fixes.
- Common: line 87
Before starting any phase, check whether a previous run left a checkpoint:
- Common: line 90
If it does not exist → fresh start
- Very specific: line 91
If it exists and status is "completed" → fresh start
- Very specific: line 92
If it exists and status is "in_progress" but timestamp is older than 24 hours → fresh start
- Very specific: line 93
If it exists and status is "in_progress" within 24 hours → resume
- Very specific: line 95
Read all existing refine-logs/round-*.md files and score-history.md
- Common: line 102
Before proposing anything, extract the user's immutable bottom-line problem.
- Common: line 112
If later reviewer feedback would change the problem being solved, mark that as drift and push back or adapt carefully.
- Very specific: line 114
Checkpoint: Write refine-logs/REFINE_STATE.json with {"phase": "anchor", "round": 0, "agent_id": null, "last_score": null, "last_verdict": …
- Very specific: line 120
Check papers/ and literature/ first.
- Common: line 128
If local material is insufficient, search recent top-venue/arXiv work online.
- Common: line 132
Do not stop at generic research questions.
- Common: line 143
Before locking the method, compare two candidate routes if both are plausible:
- Common: line 154
If both routes are weak, rethink the framing instead of combining them into a larger system by default.
- Common: line 170
Exact role of any frontier primitive: if you use an LLM / VLM / Diffusion / RL component, specify whether it acts as planner, teacher, …
- Common: line 174
If the method is still only described as "add a module" or "use a planner," it is not concrete enough.
- Common: line 189
Ensure one evidence block directly supports the Problem Anchor.
- Common: line 190
If complexity risk exists, include one simplification or deletion check.
- Common: line 191
If a single source/framework is central, include one necessity check showing why that choice matters.
- Distinctive: line 192
Default to 1-3 core evidence blocks and leave the full development roadmap to the evidence-development stage.
- Very specific: line 288
Checkpoint: Update refine-logs/REFINE_STATE.json with {"phase": "proposal", "round": 0, ...}.
- Very specific: line 397
STOP CONDITION: If overall score >= SCORE_THRESHOLD, verdict is READY, and there is no unresolved drift warning, skip to Phase 5.
- Distinctive: line 404
Write an Anchor Check:
- Distinctive: line 408
Write a Simplicity Check:
- Distinctive: line 412
If a frontier primitive is central, is its role still crisp and justified?
- Distinctive: line 416
If valid: sharpen the mechanism, simplify if possible, or modernize if the paper really improves.
- Distinctive: line 417
If debatable: revise, but explain your reasoning with evidence.
- Distinctive: line 418
If wrong, drifting, or over-complicating: push back with evidence from local papers and the Problem Anchor.
- Distinctive: line 428
Do not add multiple parallel contributions just to chase score.
- Distinctive: line 566
If the final verdict is not READY, still write the best current final version here.
- Distinctive: line 636
Ensure it contains the complete score evolution table using the new dimensions.
- Very specific: line 678
Large file handling: If the Write tool fails due to file size, immediately retry using Bash (cat << 'EOF' > file) to write in chunks.
- Distinctive: line 680
Anchor first, every round: Always carry forward the same Problem Anchor.
- Distinctive: line 681
One paper, one dominant contribution: Avoid multiple parallel contributions unless the paper truly needs them.
- Distinctive: line 683
Prefer reuse over invention: Start from strong existing backbones and add only what the bottleneck requires.
- Distinctive: line 687
Pushback is encouraged: If reviewer feedback causes drift or unnecessary complexity, argue back with evidence.
- Very specific: line 690
Do not fabricate evidence: Only describe expected evidence and planned source/case/argument work.
“Common” is measured: kept by at least 6 in 10 of the 8 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Research Refine: Problem-Anchored, Elegant, Frontier-Aware Plan Refinement
Refine and concretize: $ARGUMENTS
Overview
Use this skill when the research problem is already visible but the technical route is still fuzzy. The goal is not to produce a bloated proposal or a benchmark shopping list. The goal is to turn a vague direction into a problem -> focused method -> minimal validation document that is concrete enough to implement, elegant enough to feel paper-worthy, and current enough to resonate in the foundation-model era.
Four principles dominate this skill:
- Do not lose the original problem. Freeze an immutable Problem Anchor and reuse it in every round.
- The smallest adequate mechanism wins. Prefer the minimal intervention that directly fixes the bottleneck.
- One paper, one dominant contribution. Prefer one sharp thesis plus at most one supporting contribution.
- Modern leverage is a prior, not a decoration. When LLM / VLM / Diffusion / RL / distillation / inference-time scaling naturally fit the bottleneck, use them concretely. Do not bolt them on as buzzwords.
User input (PROBLEM + vague APPROACH)
-> Phase 0 (Local step): Freeze Problem Anchor
-> Phase 1 (Local step): Scan grounding papers -> identify technical gap -> choose the sharpest route -> write focused proposal
-> Phase 2 (Codex/GPT-5.4): Review for fidelity, specificity, contribution quality, and frontier leverage
-> Phase 3 (Local step): Anchor check + simplicity check -> revise method -> rewrite full proposal
-> Phase 4 (Codex, same agent): Re-evaluate revised proposal
-> Repeat Phase 3-4 until OVERALL SCORE >= 9 or MAX_ROUNDS reached
-> Phase 5: Save full history to refine-logs/
-> Optional handoff: develop the evidence base (sources / cases / statutes / argument reconstruction)Constants
- REVIEWER_MODEL =
gpt-5.4— Reviewer model used via a secondary Codex agent. - MAX_ROUNDS = 5 — Maximum review-revise rounds.
- SCORE_THRESHOLD = 9 — Minimum overall score to stop.
- OUTPUT_DIR =
refine-logs/— Directory for round files and final report. - MAX_LOCAL_PAPERS = 15 — Maximum local papers/notes to scan for grounding.
- MAX_CORE_EXPERIMENTS = 3 — Default cap for core validation blocks inside this skill.
- MAX_PRIMARY_CLAIMS = 2 — Soft cap for paper-level claims. Prefer one dominant claim plus one supporting claim.
- MAX_NEW_TRAINABLE_COMPONENTS = 2 — Soft cap for genuinely new trainable pieces. Exceed only if the paper breaks otherwise.
Override via argument if needed, e.g.
/research-refine "problem | approach" -- max rounds: 3, threshold: 9.
State Persistence (Checkpoint Recovery)
Long-running refinement sessions may fail mid-way (API timeout, context compaction, or session interruption). To avoid losing completed work, persist state to refine-logs/REFINE_STATE.json after each phase boundary:
{
"phase": "review",
"round": 1,
"agent_id": "019cd392-...",
"last_score": 6.5,
"last_verdict": "REVISE",
"status": "in_progress",
"timestamp": "2026-03-22T20:00:00"
}Write after each completed phase. On completion, set "status": "completed".
Output Structure
refine-logs/
├── REFINE_STATE.json
├── round-0-initial-proposal.md
├── round-1-review.md
├── round-1-refinement.md
├── round-2-review.md
├── round-2-refinement.md
├── ...
├── REVIEW_SUMMARY.md
├── FINAL_PROPOSAL.md
├── REFINEMENT_REPORT.md
└── score-history.mdEvery round-N-refinement.md must contain a full anchored proposal, not just incremental fixes.
Workflow
Initialization (Checkpoint Recovery)
Before starting any phase, check whether a previous run left a checkpoint:
- Check for
refine-logs/REFINE_STATE.json:- If it does not exist → fresh start
- If it exists and
statusis"completed"→ fresh start - If it exists and
statusis"in_progress"buttimestampis older than 24 hours → fresh start - If it exists and
statusis"in_progress"within 24 hours → resume
- On resume:
- Read all existing
refine-logs/round-*.mdfiles andscore-history.md - Recover
agent_idfor reviewer continuity - Resume from the next phase based on the saved
phase
- Read all existing
- On fresh start, ensure
refine-logs/exists and proceed to Phase 0.
Phase 0: Freeze the Problem Anchor
Before proposing anything, extract the user's immutable bottom-line problem. This anchor must be copied verbatim into every proposal and every refinement round.
Write:
- Bottom-line problem: What technical problem must be solved?
- Must-solve bottleneck: What specific weakness in current methods is unacceptable?
- Non-goals: What is explicitly not the goal of this project?
- Constraints: Compute, data, time, tooling, venue, deployment limits.
- Success condition: What evidence would make the user say "yes, this method addresses the actual problem"?
If later reviewer feedback would change the problem being solved, mark that as drift and push back or adapt carefully.
Checkpoint: Write refine-logs/REFINE_STATE.json with {"phase": "anchor", "round": 0, "agent_id": null, "last_score": null, "last_verdict": null, "status": "in_progress", "timestamp": "<now>"}.
Phase 1: Build the Initial Proposal
Step 1.1: Scan Grounding Material
Check papers/ and literature/ first. Read only the relevant parts needed to answer:
- What mechanism do current methods use?
- Where exactly do they fail for this problem?
- Which recent LLM / VLM / Diffusion / RL era techniques are actually relevant here?
- What training objectives, representations, or interfaces are reusable?
- What details distinguish a real method from a renamed high-level idea?
If local material is insufficient, search recent top-venue/arXiv work online. Focus on method sections, training setup, and failure modes, not just abstracts.
Step 1.2: Identify the Technical Gap
Do not stop at generic research questions. Make the gap operational:
- Current pipeline failure point: where does the baseline break?
- Why naive fixes are insufficient: larger context, more data, prompting, memory bank, or stacking more modules.
- Smallest adequate intervention: what is the least additional mechanism that could plausibly fix the bottleneck?
- Frontier-native alternative: is there a more current route using foundation-model-era primitives that better matches the bottleneck?
- Core technical claim: what exact mechanism claim could survive top-venue scrutiny?
- Required evidence: what minimum proof is needed to defend that claim?
Step 1.3: Choose the Sharpest Route
Before locking the method, compare two candidate routes if both are plausible:
- Route A: Elegant minimal route — the smallest mechanism that directly targets the bottleneck.
- Route B: Frontier-native route — a more modern route that uses LLM / VLM / Diffusion / RL / distillation / inference-time scaling only if it gives a cleaner or stronger story.
Then decide:
- Which route is more likely to become a strong paper under the stated constraints?
- Which route has the cleaner novelty story relative to the closest work?
- Which route avoids contribution sprawl?
If both routes are weak, rethink the framing instead of combining them into a larger system by default.
Step 1.4: Concretize the Method First
The proposal must answer "how would we actually build this?" Prefer method detail over broad experimentation and prefer reuse over invention.
Cover:
- One-sentence method thesis: the single strongest mechanism claim.
- Contribution focus: one dominant contribution and at most one supporting contribution.
- Complexity budget: what is frozen or reused, what is new, and what tempting additions are intentionally excluded.
- System graph: modules, data flow, inputs, outputs.
- Representation design: what latent, embedding, plan token, reward signal, memory state, or alignment space is used?
- Training recipe: data source, supervision, pseudo-labeling, negatives, curriculum, losses, weighting, stagewise vs joint training.
- Inference path: how the trained components are used at test time and what signals flow where.
- Why the mechanism stays small: why a larger stack is unnecessary.
- Exact role of any frontier primitive: if you use an LLM / VLM / Diffusion / RL component, specify whether it acts as planner, teacher, critic, reward model, generator prior, search controller, or distillation source.
- Failure handling: what could go wrong and what fallback or diagnostic exists?
- Novelty and elegance argument: why this is more than naming a module and why the paper still looks focused.
If the method is still only described as "add a module" or "use a planner," it is not concrete enough.
Step 1.5: Design Minimal Claim-Driven Validation
Experiments exist to validate the method, not to dominate the document.
For each core claim, define the smallest strong experiment that can validate it:
- the claim being tested
- the necessary baseline or ablation
- the decisive metric
- the expected directional outcome
Additional rules:
- Ensure one evidence block directly supports the Problem Anchor.
- If complexity risk exists, include one simplification or deletion check.
- If a single source/framework is central, include one necessity check showing why that choice matters.
- Default to 1-3 core evidence blocks and leave the full development roadmap to the evidence-development stage.
Step 1.6: Write the Initial Proposal
Save to refine-logs/round-0-initial-proposal.md.
Use this structure:
# Research Proposal: [Title]
## Problem Anchor
- Bottom-line problem:
- Must-solve bottleneck:
- Non-goals:
- Constraints:
- Success condition:
## Technical Gap
[Why current methods fail, why naive bigger systems are not enough, and what mechanism is missing]
## Method Thesis
- One-sentence thesis:
- Why this is the smallest adequate intervention:
- Why this route is timely in the foundation-model era:
## Contribution Focus
- Dominant contribution:
- Optional supporting contribution:
- Explicit non-contributions:
## Proposed Method
### Complexity Budget
- Frozen / reused backbone:
- New trainable components:
- Tempting additions intentionally not used:
### System Overview
[Step-by-step pipeline or ASCII graph]
### Core Mechanism
- Input / output:
- Architecture or policy:
- Training signal / loss:
- Why this is the main novelty:
### Optional Supporting Component
- Only include if truly necessary:
- Input / output:
- Training signal / loss:
- Why it does not create contribution sprawl:
### Modern Primitive Usage
- Which LLM / VLM / Diffusion / RL-era primitive is used:
- Exact role in the pipeline:
- Why it is more natural than an old-school alternative:
### Integration into Base Generator / Downstream Pipeline
[Where the new method attaches, what is frozen, what is trainable, inference order]
### Training Plan
[Stagewise or joint training, losses, data construction, pseudo-labels, schedules]
### Failure Modes and Diagnostics
- [Failure mode]:
- [How to detect]:
- [Fallback or mitigation]:
### Novelty and Elegance Argument
[Closest work, exact difference, why this is a focused mechanism-level contribution rather than a module pile-up]
## Claim-Driven Validation Sketch
### Claim 1: [Main claim]
- Minimal experiment:
- Baselines / ablations:
- Metric:
- Expected evidence:
### Claim 2: [Optional]
- Minimal experiment:
- Baselines / ablations:
- Metric:
- Expected evidence:
## Experiment Handoff Inputs
- Must-prove claims:
- Must-run ablations:
- Critical datasets / metrics:
- Highest-risk assumptions:
## Compute & Timeline Estimate
- Estimated GPU-hours:
- Data / annotation cost:
- Timeline:Checkpoint: Update refine-logs/REFINE_STATE.json with {"phase": "proposal", "round": 0, ...}.
Phase 2: External Method Review (Round 1)
Send the full proposal to GPT-5.4 for an elegance-first, frontier-aware, method-first review. The reviewer should spend most of the critique budget on the method itself, not on expanding the experiment menu.
spawn_agent:
model: REVIEWER_MODEL
reasoning_effort: xhigh
message: |
You are a senior ML reviewer for a top venue (NeurIPS/ICML/ICLR).
This is an early-stage, method-first research proposal.
Your job is NOT to reward extra modules, contribution sprawl, or a giant benchmark checklist.
Your job IS to stress-test whether the proposed method:
(1) still solves the original anchored problem,
(2) is concrete enough to implement,
(3) presents a focused, elegant contribution,
(4) uses foundation-model-era techniques appropriately when they are the natural fit.
Review principles:
- Prefer the smallest adequate mechanism over a larger system.
- Penalize parallel contributions that make the paper feel unfocused.
- If a modern LLM / VLM / Diffusion / RL route would clearly produce a better paper, say so concretely.
- If the proposal is already modern enough, do NOT force trendy components.
- Do not ask for extra experiments unless they are needed to prove the core claims.
Read the Problem Anchor first. If your suggested fix would change the problem being solved,
call that out explicitly as drift instead of treating it as a normal revision request.
=== PROPOSAL ===
[Paste the FULL proposal from Phase 1]
=== END PROPOSAL ===
Score these 7 dimensions from 1-10:
1. **Problem Fidelity**: Does the method still attack the original bottleneck, or has it drifted into solving something easier or different?
2. **Method Specificity**: Are the interfaces, representations, losses, training stages, and inference path concrete enough that an engineer could start implementing?
3. **Contribution Quality**: Is there one dominant mechanism-level contribution with real novelty, good parsimony, and no obvious contribution sprawl?
4. **Frontier Leverage**: Does the proposal use current foundation-model-era primitives appropriately when they are the right tool, instead of defaulting to old-school module stacking?
5. **Feasibility**: Can this method be trained and integrated with the stated resources and data assumptions?
6. **Validation Focus**: Are the proposed experiments minimal but sufficient to validate the core claims? Is there unnecessary experimental bloat?
7. **Venue Readiness**: If executed well, would the contribution feel sharp and timely enough for a top venue?
**OVERALL SCORE** (1-10): Weighted toward Problem Fidelity, Method Specificity, Contribution Quality, and Frontier Leverage.
Use this weighting: Problem Fidelity 15%, Method Specificity 25%, Contribution Quality 25%, Frontier Leverage 15%, Feasibility 10%, Validation Focus 5%, Venue Readiness 5%.
For each dimension scoring < 7, provide:
- The specific weakness
- A concrete fix at the method level (interface / loss / training recipe / integration point / deletion of unnecessary parts)
- Priority: CRITICAL / IMPORTANT / MINOR
Then add:
- **Simplification Opportunities**: 1-3 concrete ways to delete, merge, or reuse components while preserving the main claim. Write "NONE" if already tight.
- **Modernization Opportunities**: 1-3 concrete ways to replace old-school pieces with more natural foundation-model-era primitives if genuinely better. Write "NONE" if already modern enough.
- **Drift Warning**: "NONE" if the proposal still solves the anchored problem; otherwise explain the drift clearly.
- **Verdict**: READY / REVISE / RETHINK
Verdict rule:
- READY: overall score >= 9, no meaningful drift, one focused dominant contribution, and no obvious complexity bloat remains
- REVISE: the direction is promising but not yet at READY bar
- RETHINK: the core mechanism or framing is still fundamentally offCRITICAL: Save the agent_id from this call for all later rounds.
CRITICAL: Save the FULL raw response verbatim.
Save review to refine-logs/round-1-review.md with the raw response in a <details> block.
Checkpoint: Update refine-logs/REFINE_STATE.json with {"phase": "review", "round": 1, "agent_id": "<saved>", "last_score": <parsed>, "last_verdict": "<parsed>", ...}.
Phase 3: Parse Feedback and Revise the Method
Step 3.1: Parse the Review
Extract:
- Problem Fidelity
- Method Specificity
- Contribution Quality
- Frontier Leverage
- Feasibility
- Validation Focus
- Venue Readiness
- Overall score
- Verdict
- Drift Warning
- Simplification Opportunities
- Modernization Opportunities
- Action items ranked by priority
Update refine-logs/score-history.md:
# Score Evolution
| Round | Problem Fidelity | Method Specificity | Contribution Quality | Frontier Leverage | Feasibility | Validation Focus | Venue Readiness | Overall | Verdict |
|-------|------------------|--------------------|----------------------|-------------------|-------------|------------------|-----------------|---------|---------|
| 1 | X | X | X | X | X | X | X | X | REVISE |STOP CONDITION: If overall score >= SCORE_THRESHOLD, verdict is READY, and there is no unresolved drift warning, skip to Phase 5.
Step 3.2: Revise With an Anchor Check and a Simplicity Check
Before changing anything:
- Copy the Problem Anchor verbatim.
- Write an Anchor Check:
- What is the original bottleneck?
- Does the current method still solve it?
- Which reviewer suggestions would cause drift if followed blindly?
- Write a Simplicity Check:
- What is the dominant contribution now?
- What components can be removed, merged, or kept frozen?
- Which reviewer suggestions add unnecessary complexity?
- If a frontier primitive is central, is its role still crisp and justified?
Then process reviewer feedback:
- If valid: sharpen the mechanism, simplify if possible, or modernize if the paper really improves.
- If debatable: revise, but explain your reasoning with evidence.
- If wrong, drifting, or over-complicating: push back with evidence from local papers and the Problem Anchor.
Bias the revisions toward:
- a sharper central contribution
- fewer moving parts
- cleaner reuse of strong existing backbones
- more natural foundation-model-era leverage when it improves the paper
- leaner, claim-driven experiments
Do not add multiple parallel contributions just to chase score. If the reviewer requests another module, first ask whether the same gain can come from a better interface, distillation signal, reward model, or inference policy on top of an existing backbone.
Save to refine-logs/round-N-refinement.md:
# Round N Refinement
## Problem Anchor
[Copy verbatim from round 0]
## Anchor Check
- Original bottleneck:
- Why the revised method still addresses it:
- Reviewer suggestions rejected as drift:
## Simplicity Check
- Dominant contribution after revision:
- Components removed or merged:
- Reviewer suggestions rejected as unnecessary complexity:
- Why the remaining mechanism is still the smallest adequate route:
## Changes Made
### 1. [Method section changed]
- Reviewer said:
- Action:
- Reasoning:
- Impact on core method:
### 2. [Novelty / modernity / feasibility / validation change]
- Reviewer said:
- Action:
- Reasoning:
- Impact on core method:
## Revised Proposal
[Full updated proposal from Problem Anchor through Claim-Driven Validation Sketch]Checkpoint: Update refine-logs/REFINE_STATE.json with {"phase": "refine", "round": N, ...}.
Phase 4: Re-evaluation (Round 2+)
Send the revised proposal back to GPT-5.4 in the same agent:
send_input:
id: [saved from Phase 2]
model: REVIEWER_MODEL
reasoning_effort: xhigh
message: |
[Round N re-evaluation]
I revised the proposal based on your feedback.
First, check whether the original Problem Anchor is still preserved.
Second, judge whether the method is now more concrete, more focused, and more current.
Key changes:
1. [Method change 1]
2. [Method change 2]
3. [Simplification / modernization / pushback if any]
=== REVISED PROPOSAL ===
[Paste the FULL revised proposal]
=== END REVISED PROPOSAL ===
Please:
- Re-score the same 7 dimensions and overall
- State whether the Problem Anchor is preserved or drifted
- State whether the dominant contribution is now sharper or still too broad
- State whether the method is simpler or still overbuilt
- State whether the frontier leverage is now appropriate or still old-school / forced
- Focus new critiques on missing mechanism, weak training signal, weak integration point, pseudo-novelty, or unnecessary complexity
- Use the same verdict rule: READY only if overall score >= 9 and no blocking issue remains
Same output format: 7 scores, overall score, verdict, drift warning, simplification opportunities, modernization opportunities, remaining action items.Save review to refine-logs/round-N-review.md.
Checkpoint: Update refine-logs/REFINE_STATE.json with {"phase": "review", "round": N, "agent_id": "<saved>", "last_score": <parsed>, "last_verdict": "<parsed>", ...}.
Then return to Phase 3 until:
- Overall score >= SCORE_THRESHOLD and verdict is READY and no unresolved drift
- or MAX_ROUNDS reached
Phase 5: Final Report and Logs
Step 5.1: Write refine-logs/REVIEW_SUMMARY.md
This file is the high-level round-by-round review record. It should answer: each round was trying to solve what, what changed, what got resolved, and what remained.
# Review Summary
**Problem**: [user's problem]
**Initial Approach**: [user's vague approach]
**Date**: [today]
**Rounds**: N / MAX_ROUNDS
**Final Score**: X / 10
**Final Verdict**: [READY / REVISE / RETHINK]
## Problem Anchor
[Verbatim anchor used across all rounds]
## Round-by-Round Resolution Log
| Round | Main Reviewer Concerns | What This Round Simplified / Modernized | Solved? | Remaining Risk |
|-------|-------------------------|------------------------------------------|---------|----------------|
| 1 | [top issues from review] | [main method changes] | [yes / partial / no] | [if any] |
| 2 | ... | ... | ... | ... |
## Overall Evolution
- [How the method became more concrete]
- [How the dominant contribution became more focused]
- [How unnecessary complexity was removed]
- [How modern technical leverage improved or stayed intentionally minimal]
- [How drift was avoided or corrected]
## Final Status
- Anchor status: [preserved / corrected / unresolved]
- Focus status: [tight / slightly broad / still diffuse]
- Modernity status: [appropriately frontier-aware / intentionally conservative / still old-school]
- Strongest parts of final method:
- Remaining weaknesses:Step 5.2: Write refine-logs/FINAL_PROPOSAL.md
This file is the clean final version document. It should contain only the final proposal itself, without review chatter, round history, or raw reviewer output.
# Research Proposal: [Title]
[Paste the final refined proposal only]If the final verdict is not READY, still write the best current final version here.
Step 5.3: Write refine-logs/REFINEMENT_REPORT.md
# Refinement Report
**Problem**: [user's problem]
**Initial Approach**: [user's vague approach]
**Date**: [today]
**Rounds**: N / MAX_ROUNDS
**Final Score**: X / 10
**Final Verdict**: [READY / REVISE / RETHINK]
## Problem Anchor
[Verbatim anchor used across all rounds]
## Output Files
- Review summary: `refine-logs/REVIEW_SUMMARY.md`
- Final proposal: `refine-logs/FINAL_PROPOSAL.md`
## Score Evolution
| Round | Problem Fidelity | Method Specificity | Contribution Quality | Frontier Leverage | Feasibility | Validation Focus | Venue Readiness | Overall | Verdict |
|-------|------------------|--------------------|----------------------|-------------------|-------------|------------------|-----------------|---------|---------|
| 1 | ... | ... | ... | ... | ... | ... | ... | ... | ... |
## Round-by-Round Review Record
| Round | Main Reviewer Concerns | What Was Changed | Result |
|-------|-------------------------|------------------|--------|
| 1 | [top issues] | [main fixes] | [resolved / partial / unresolved] |
| 2 | ... | ... | ... |
## Final Proposal Snapshot
- Canonical clean version lives in `refine-logs/FINAL_PROPOSAL.md`
- Summarize the final thesis in 3-5 bullets here
## Method Evolution Highlights
1. [Most important simplification or focusing move]
2. [Most important mechanism upgrade]
3. [Most important modernization or justification for staying simple]
## Pushback / Drift Log
| Round | Reviewer Said | Author Response | Outcome |
|-------|---------------|-----------------|---------|
| 1 | [criticism] | [pushback + anchor / evidence] | [accepted / rejected] |
## Remaining Weaknesses
[Honest unresolved issues]
## Raw Reviewer Responses
<details>
<summary>Round 1 Review</summary>
[Full verbatim response from GPT-5.4]
</details>
...
## Next Steps
- If READY: proceed to develop the evidence base (`/statute-case-mapper`, `/comparative-law`, `/argument-stress-test`, or source/corpus work)
- If REVISE: manually address the remaining argument weaknesses, then re-run `/research-refine`
- If RETHINK: revisit the core thesis, possibly re-surveying with `/research-lit`Step 5.4: Finalize score-history.md
Ensure it contains the complete score evolution table using the new dimensions.
Step 5.5: Present a Brief Summary to the User
Refinement complete after N rounds.
Final score: X/10 (Verdict: READY / REVISE / RETHINK)
Anchor status:
- [preserved / drift corrected / unresolved concern]
Focus status:
- [tight / slightly broad / still diffuse]
Modernity status:
- [appropriately frontier-aware / intentionally conservative / still old-school]
Key method upgrades:
- [method change 1]
- [method change 2]
Remaining concerns:
- [if any]
Review summary: refine-logs/REVIEW_SUMMARY.md
Full report: refine-logs/REFINEMENT_REPORT.md
Final proposal: refine-logs/FINAL_PROPOSAL.md
Suggested next step: develop the evidence baseCheckpoint: Update refine-logs/REFINE_STATE.json with {"phase": "done", "status": "completed", ...}.
Output Protocols
Follow these shared protocols for all output files:
- Output Versioning Protocol — write timestamped file first, then copy to fixed name
- Output Manifest Protocol — log every output to MANIFEST.md
- Output Language Protocol — respect the project's language setting
Key Rules
- Large file handling: If the Write tool fails due to file size, immediately retry using Bash (
cat << 'EOF' > file) to write in chunks. Do NOT ask the user for permission — just do it silently. - Anchor first, every round. Always carry forward the same Problem Anchor.
- One paper, one dominant contribution. Avoid multiple parallel contributions unless the paper truly needs them.
- The smallest adequate mechanism wins. Bigger is not automatically better.
- Prefer reuse over invention. Start from strong existing backbones and add only what the bottleneck requires.
- Frameworks are a prior, not a decoration. Adopt a theoretical framework or method when it sharpens the argument, not when it only makes the proposal sound sophisticated.
- Minimal evidence. Inside this skill, the evidence blocks only need to prove the core claims.
- Review the argument, not the citation count. A long source list is not novelty.
- Pushback is encouraged. If reviewer feedback causes drift or unnecessary complexity, argue back with evidence.
- ALWAYS use
reasoning_effort: xhighfor all Codex review calls. - Save
agent_idfrom Phase 2 and usesend_inputfor later rounds. - Do not fabricate evidence. Only describe expected evidence and planned source/case/argument work.
- Be specific about source and method assumptions. Vague "we'll cite some cases" is not enough.
- Document everything. Save every raw review, every anchor check, every simplicity check, and every major method change.
Composing with Other Skills
This skill sits between literature review and evidence development:
/research-refine-pipeline -> one-shot refine + evidence planning
/research-lit "direction" -> survey the literature
/research-refine "PROBLEM: ... | THESIS: ..." <- you are here
develop evidence -> statute-case-mapper / comparative-law / argument-stress-test / source work
/research-review -> iterate on the draft and argumentTypical flow:
/research-litor local reading gives you a problem and a vague thesis direction/research-refineturns that into an anchored, elegant, well-grounded thesis plan- the evidence-development stage turns the final proposal into a detailed claim-driven evidence roadmap
/research-refine-pipelineis the one-shot wrapper when the user wants both stages in a single request- evidence development builds the sources, cases, and arguments
- Later loops operate on the developed evidence, not just the thesis
This skill also works standalone if you already know the problem and just need the method to become concrete.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

