agentleFS
xzljqxzljq/claude-skills/skills/research-ideation/SKILL.md

research-ideation

Skill0 starsChanged 3 months agoMITView on GitHub

Generate and evaluate new research ideas on top of a verified literature base, with Codex producing candidates independently and in parallel. Use whenever the user proposes, requests, or evaluates a new research idea or direction.

15 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use results-report, daily-coding and 10 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • Galaxy-DawnSkill

    results-report

    This skill should be used when the user asks to "write an experiment report", "summarize experimental results", "do experiment retrospection", "write a results report", "写实验总结报告", "写实验复盘", or mentions turning completed experiment artifacts into a structured, decision-oriented research report. It assumes strict analysis should come from `results-analysis` first.

    Also in 3 projects that use this

  • Clay-HHKSkill

    daily-coding

    Daily coding assistant that auto-triggers when writing/modifying code, providing a core checklist.

    Also in 3 projects that use this

  • Clay-HHKSkill

    kaggle-learner

    This skill should be used when the user asks to "learn from Kaggle", "study Kaggle solutions", "analyze Kaggle competitions", or mentions Kaggle competition URLs. Provides access to extracted knowledge from winning Kaggle solutions across NLP, CV, time series, tabular, and multimodal domains.

    Also in 3 projects that use this

  • kharlamenkodevSkill

    bug-detective

    Also in 3 projects that use this

  • Google CloudSkill

    citation-verification

    Verify that citations in a document resolve to real records and that claimed titles match. Use before trusting a document's reference list.

    Also in 4 projects that use this

  • AnthropicSkill

    command-development

    This skill should be used when the user asks to "create a slash command", "add a command", "write a custom command", "define command arguments", "use command frontmatter", "organize commands", "create command with file references", "interactive command", "use AskUserQuestion in command", or needs guidance on slash command structure, YAML frontmatter fields, dynamic arguments, bash execution in commands, user interaction patterns, or command development best practices for Claude Code.

    Also in 3 projects that use this

  • +2 more from Clay-HHK

The file

We split this skill into the rules it gives an agent

7 rules. 0 are common, 6 are distinctive, and 1 is very specific to this project.

  • 0

    Common

    Most of the 14 edited versions we read keep these.

  • 6

    Distinctive

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Keep the full union in the record.

    line 87
  2. Distinctive:

    Use ordinal judgments with reasons unless calibrated estimates exist.

    line 116
  3. Distinctive:

    Run full mode ② checks on at most five candidates at a time.

    line 120
  4. Distinctive:

    Every candidate admitted to the novelty-check budget goes through

    line 155
  5. Distinctive:

    If a simple way to game a proposed metric is already visible at design time, the metric is wrong for that use — change it rather than plan …

    line 215
  6. Distinctive:

    Execution: call Codex MCP tools from the MAIN conversation, never from a Claude subagent.

    line 227
  7. Very specific:

    Never send Claude's analysis first.

    line 230

“Common” is measured: kept by at least 6 in 10 of the 14 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Research ideation

Ends at a ranked set of promising directions whose novelty has been checked. Experiment design is deliberately out of scope — find a direction worth pursuing first, then go deep via codex-collab.

Hard prerequisite

research-lit-survey mode ① must be completed first, including full-text reading of the inspiration sources.

Do not search and ideate at the same time. Framing formed during an exploratory search gets locked in by whatever the early (often wrong) information said, and later corrections do not undo it. Establish the verified ledger, then ideate.

Inspiration sources are restricted to Tier A/B. Recent papers are the richest inspiration source — they carry angles nobody has worked out yet — but Tier-C reliability is too low to build on.


Procedure

Step 1 — Evidence and prior-art maps

From the ledger, build:

  • Credible evidence map: one row per sub-question, with the Tier-A/B works that may support field-state or inspiration claims and their tier.
  • Known prior-art map: one row per sub-question or mechanism, with every public work that occupies it, regardless of tier.
  • Provisional gaps: what no known public work covers. These remain provisional until mode ② actively searches for prior art.

Tier controls how a work may be used as evidence or inspiration; it never controls whether the work occupies novelty. A Tier-C work may not inspire the idea, but it can still defeat or narrow a novelty claim.

Step 2 — Independent parallel candidate generation

Purpose: widen the candidate space. Two models with different training data and different instincts propose different things; the union is the pool. Independence here is in service of breadth — it is not a voting procedure.

Claude and Codex generate candidates independently and in parallel.

  • Codex runs in a FRESH thread.
  • Codex receives only: the research question, constraints (compute, scale, main metric, venue target), and the verified literature boundary from the ledger.
  • Codex receives none of Claude's ideas, leanings, or preliminary conclusions.
  • Include in the Codex prompt: "Do not ask clarification questions unless the task is impossible to interpret. When information is underspecified, state your assumptions explicitly and proceed independently. The purpose of this call is to obtain an independent research perspective, not to converge interactively."
  • Codex's tools remain available, but during this generation step it must not perform new literature retrieval or landscape verification. It ideates only from the supplied verified literature boundary; retrieval resumes in Step 4.
  • Claude reasons through the problem independently while the Codex call runs.

Step 3 — Merge into a candidate pool

Take the union, with attribution preserved: which side proposed what, and where both landed on the same thing.

Do not prune by convergence at this stage. An idea both models produced independently is, if anything, the more obvious one — convergence at generation time tracks salience, not quality. An idea only one side produced may well be the more original candidate. Filtering on agreement here would systematically discard the novelty the step exists to find.

Attribution is carried forward because it is informative later, not because it ranks anything now.

Keep the full union in the record. Before the expensive novelty check, run one pre-novelty screen with two layers. The layers serve different purposes: hard gates remove invalid candidates; triage ranks the survivors.

Layer A — hard gates. Exclude a candidate only if:

  • its core claim is not falsifiable;
  • it has neither a main-metric redemption chain nor a diagnostic endpoint that could change practice;
  • its proposed metric has an obvious gaming path;
  • it violates the stated constraints, including having no discriminating experiment that is possible under them; or
  • it is strictly dominated by a cheaper and stronger formulation.

Record every exclusion and its reason; never label these candidates "not novel." A candidate that is merely uncertain or expensive survives this layer.

Layer B — low-cost triage. For every survivor, make a rough, evidence-labeled assessment of:

  • the scientific payoff if the claim is true, including the minimum worthwhile movement on a field-recognized main metric; for diagnostic work, use the importance of the failure and the strength of the actionable implication;
  • mechanism plausibility;
  • whether the cheapest discriminating experiment would test the full redemption chain rather than an easy proxy;
  • the likelihood of reaching the minimum worthwhile effect; and
  • the time and compute needed to obtain the first decisive evidence.

Use ordinal judgments with reasons unless calibrated estimates exist. These are ranking factors, not additional pass/fail gates. The detailed feasibility assessment remains in Step 5.

Run full mode ② checks on at most five candidates at a time. If more than five survive, build a provisional shortlist of three to five. Allocate the novelty-check budget in three stages; these are sequential decisions, not scores to average.

Stage 1 — prior-art threshold. Drop a candidate from budget contention only when the Step 1 literature boundary already shows that its core mechanism and claimed contribution are occupied. Close work alone is not enough: narrow the claim first and retain the candidate if a meaningful gap remains. Record the decision without calling it a completed novelty check.

Stage 2 — rank by decision value of checking. Rank candidates by how much a mode ② check is expected to change the pursue/defer decision, relative to the cost of that check. The candidate's conditional scientific payoff and the Layer-B feasibility triage determine how consequential that decision would be; current uncertainty about prior art determines how likely the check is to change it. Do not substitute redemption-chain strength × P(uncovered) as a final idea score.

Stage 3 — apply set-level constraints. Diversity is a property of the chosen set, not a score on any single candidate: normally cap one mechanism family at two slots, so the budget spans distinct hypotheses rather than variants of one. Treat this as a portfolio default, not a reason to override clear dominance; state any exception.

Present the provisional shortlist with a one-line reason for each candidate and list all deferred survivors in ranked order. If candidates differ materially in research direction, risk/payoff, or compute commitment, stop for the user's choice before Step 4. Proceed automatically only when the ordering follows from clear dominance and does not choose a new research direction. Deferred candidates stay in the ledger as "not novelty-checked", eligible for a later round.

Step 4 — Novelty check (mandatory)

Every candidate admitted to the novelty-check budget goes through research-lit-survey mode ②. An idea that has not been through the novelty check may not be presented as novel or included in the final ranked set. This is the first novelty filter on the pool.

Step 5 — Full joint feasibility assessment and final ranking

The Step 3 triage is deliberately cheap and provisional. Once novelty checking has narrowed the pool, this is where two models are used for confirmation rather than breadth. For each surviving candidate, have both Claude and Codex reassess it using the verified closest prior art and a concrete proposed test: is the mechanism plausible, would the minimal experiment actually falsify the full redemption chain, is a worthwhile main-metric effect plausible, what will the first decisive evidence cost, and what is the most likely non-result.

Agreement is meaningful here in a way it is not at Step 3, because the claim being judged is specific rather than open-ended. Disagreement here is a substantive signal about feasibility and goes to the user (Step 7).

Step 6 — Deliverable format

Per idea:

fieldcontent
Core claimone falsifiable sentence
Why not coveredclosest prior art across all tiers, the precise difference, and known coverage limitations
Minimal falsificationthe cheapest experiment that could kill it
Main metrica field-recognized one; custom metrics are mechanism-only
Biggest riskthe most likely way it turns out to be a non-result
Novelty score / P(main metric moves)honest estimates, not advocacy

Also state, for the set as a whole, which single idea to bet on and why, and which nearby ideas are traps (saturated, or likely to yield an unpublishable negative result).

Step 7 — Route decisions to the user

Disagreements between Claude and Codex that affect experimental conclusions, result credibility, or direction are key decisions: present both positions with trade-offs via AskUserQuestion. Do not self-arbitrate.

Which direction to pursue is always the user's call.


Standards an idea must meet

Inherited from the user's research methodology — restated because ideation is where they get violated:

  • Method work succeeds only by improving a field-recognized main metric under fair, comparable, seed-verified settings. Custom intermediate metrics explain mechanism; they never substitute for the main metric.
  • Diagnostic work is acceptable as a direction only if it reaches the level of changing practice — overturning a common practice, explaining a known failure, or yielding a verified actionable inference. Observing that some intermediate quantity moves is not enough.
  • A new problem or metric must come with the redemption chain written down: which already-accepted metric it ultimately cashes out to.
  • If a simple way to game a proposed metric is already visible at design time, the metric is wrong for that use — change it rather than plan to audit around it later.

Shared red lines

<!-- CANONICAL BLOCK — verbatim copy. Source of truth: codex-collab/SKILL.md. Duplicated here and in research-lit-survey so loading one skill suffices. Any edit must be applied to all three files identically. -->
  • Execution: call Codex MCP tools from the MAIN conversation, never from a Claude subagent. Long calls get backgrounded — do useful work meanwhile.
  • Independence: for an independent opinion, always start a FRESH thread. Never send Claude's analysis first. Use codex-reply only after the independent first response has arrived.
  • Tool access: Codex has web access and its own tools. Never write "do not run shell commands" or "answer from your own knowledge" into a prompt for a task that needs retrieval — Codex will then report that it did not search, which reflects the prompt's constraints, not its actual access. Require resolvable citations and verify them. The MCP call's own parameters impose the same constraint silently, and this is the easier mistake to make: sandbox: read-only blocks network at the shell layer, workspace-write leaves network_access off by default, and approval-policy: never removes any path to escalate. A retrieval task launched that way is crippled no matter how carefully the prompt is worded, and it usually degrades partially — Codex still answers, from a narrower channel — rather than failing loudly. For any task that must retrieve, pass sandbox: danger-full-access. Treat a Codex report of blocked access as a launch-configuration bug to fix and relaunch; that relaunch is a corrected call, not the one retry allowed under Timeout. Its "I could not find X" is never evidence of absence.
  • Deliverable transport: the MCP result returns only Codex's final turn, so a long answer delivered as chat in an earlier turn is silently lost. For anything substantial, instruct Codex to write its deliverable to a named file and reply with just the path. If a result comes back looking like housekeeping, a stray fragment, or an off-topic note, the real work probably exists on disk — go look before concluding it failed or relaunching it.
  • Clarification routing: Codex handles its own underspecification by stating assumptions (L1); Claude relays only facts already stated by the user or already measured (L2); ask the user only when the missing information changes the research question, scientific assumptions, or acceptable tradeoffs (L3). Never invent facts or user preferences to satisfy Codex.
  • Timeout: slow ≠ failed. On genuine failure retry at most ONCE in a fresh thread; if that fails too, proceed with Claude's own analysis and state explicitly that independent validation was unavailable.
  • Disagreement: Claude/Codex disagreements that affect experimental conclusions, result credibility, or direction are key decisions — surface both positions with trade-offs via AskUserQuestion. Do not self-arbitrate.
<!-- END CANONICAL BLOCK -->

Ideation-specific addendum (outside the canonical block): in Step 2, Codex must not ask clarification questions at all — assumptions plus an independent answer only.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.