research-critique
Skill340 starsChanged 8 months agoMITView on GitHub
Critical analysis of research papers evaluating methodology, claims-evidence alignment, and contribution significance. Triggers on: "critique this paper", "review this research", "analyze this study", "evaluate the methodology", "is this paper sound". NOT for formatting or submission readiness, use manuscript-review.
1 other project uses a version of this.
Adopted by
Most recently taken up by
- GRIDLOCK-NYC/claude-skillstheir own version, first seen Oct 8, 2026
Other versions1 project uses a version of this.
Word for word · 0
None found.
Edited · 0
None found.
Wrote their own version · 1
- GRIDLOCK-NYC/claude-skills★ 0same name
How matches are found. Word for word: the exact same text. Edited: shares passages with this file (“99% same” means 99% of its five-word phrases also appear here; 10% or more counts). Their own version: similar meaning, or, weakest, the same name. We can’t tell which project had it first. Projects are counted among the 200 most-starred candidates, and mentions are links to the repository we found, up to 50 per site, so both counts are a floor.
In this repository94 other files it gives its agents.
- AGENTS.mdAGENTS.md
- adr-writerSkill
- agent-builderSkill
- animateSkill
- api-docs-generatorSkill
- appraiseSkill
- architecture-diagramSkill
- architecture-reviewerSkill
- arxiv-figuresSkill
- arxiv-packageSkill
- arxiv-preflightSkill
- benchmark-runnerSkill
- changelog-composerSkill
- chart-claritySkill
- citation-auditSkill
- codebase-advisorSkill
- code-refinerSkill
- competitive-analyzerSkill
- concept-to-imageSkill
- debug-investigatorSkill
- decision-mapSkill
- dependency-auditSkill
- devils-advocateSkill
- domain-modelingSkill
- engineering-retroSkill
- env-validatorSkill
- estimate-calibratorSkill
- feasibility-assessorSkill
- figure-rhetoricSkill
- figure-table-qualitySkill
- filesystemSkill
- githubSkill
- gpu-optimizerSkill
- handoffSkill
- html-presentationSkill
- html-videoSkill
- humanizeSkill
- idea-validatorSkill
- immuneSkill
- intakeSkill
- kg-builderSkill
- launch-videoSkill
- learning-workspaceSkill
- lightpanda-browserSkill
- linkedin-post-styleSkill
- literature-reviewSkill
- manuscript-provenanceSkill
- manuscript-reviewSkill
- manuscript-typographySkill
- market-analyzerSkill
- marp-slidesSkill
- mcp-to-skillSkill
- md-to-pdfSkill
- migration-risk-analyzerSkill
- milestone-runnerSkill
- module-deepeningSkill
- notebooklmSkill
- package-evaluatorSkill
- package-optimizerSkill
- paper-to-skillSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
4 rules. About 0 are common, 4 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈4
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 28
Read the paper as the author intended it.
- Distinctive: line 62
Do not produce a numbered checklist of weaknesses.
- Distinctive: line 86
Before finalizing your critique, verify each point against these filters:
- Distinctive: line 96
Not every input is a full empirical paper.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Research Paper Critique
Purpose
Critically evaluate research papers as an engaged, intellectually honest colleague — not an adversarial reviewer generating decorative objections. The goal is understanding what a paper contributes, whether the evidence supports the claims, and where genuine tension exists between what is demonstrated and what is argued.
The failure mode you must avoid
The default behavior when critiquing a paper is adversarial credentialism: treating the paper as something to defeat, generating a volume of objections to demonstrate rigor, and mistaking the enumeration of limitations for insight. This produces critiques that are shallow, performative, and often incoherent — holding papers to standards that no published work in the field meets, demanding ablations that would constitute separate papers, and flagging limitations the author already disclosed as if discovering them.
The tell: if you retreat easily from a point when challenged, you were never committed to it. It was decorative, not substantive. Do not generate points you would abandon under pressure.
Analytical principles
1. Understand before you evaluate
Read the paper as the author intended it. What problem does it identify? What is the proposed mechanism or contribution? What is the experimental design trying to isolate? What claims does the paper actually make, versus what you assume it is claiming?
Most bad critiques attack a paper the author did not write. Get the real paper straight first.
2. Proportionality
Every empirical paper has limitations. Uncontrolled confounds, limited domains, incomplete ablations, finite sample sizes. These exist in every paper ever published, including the landmarks of the field. Listing them is not critique. The question is always: does this limitation actually undermine the paper's specific contribution?
A limitation that the author discloses, contextualizes, and accounts for in their claims is not a flaw you discovered. A limitation that would require a different experiment entirely to resolve is not a flaw — it is future work. A limitation that exists in every comparable study is not a differentiating weakness.
Weight your critique by consequence. A confound that could fully explain the main result matters. A confound that affects a secondary finding the paper does not emphasize does not.
3. No phantom standards
Do not hold the paper against an imaginary ideal that nothing in the field achieves. Before asserting that the paper should have done X, ask: does any comparable work do X? If the answer is no, then you are requesting a methodological advance, not identifying a flaw. Name it as such.
The implicit comparison should be the realistic state of the field, not a platonic ideal of experimental design.
4. Mechanism versus evidence
Many papers propose a mechanism (why something works) and provide evidence (that it works). These are different claims with different standards. The evidence can be solid while the mechanistic story is underdetermined. This is normal and nearly universal in empirical work.
When critiquing the mechanism: identify what alternative explanations the data cannot distinguish between. Do not substitute your preferred alternative mechanism and assert it is "likely" doing the work — that is the same sin as the paper's, just pointing the other direction. If you cannot separate two explanations, say so honestly rather than privileging one.
5. Scope is not a weakness
A paper that tests in one domain and theorizes about generality is doing what papers do. The theory predicts what should happen elsewhere. Someone else can test it. Demanding that a single paper demonstrate generality across all domains it might apply to is demanding a research program, not a paper. Critique the scope if the paper overclaims relative to its evidence. Do not critique it for having a scope.
6. What survives
After your analysis, state clearly: what does this paper contribute that holds up? What finding, if confirmed, would the field benefit from knowing? Default to identifying the contribution, not to dismissal. A paper that advances understanding in a limited domain with honest limitations is a good paper. Say so.
Output structure
Do not produce a numbered checklist of weaknesses. Instead, write a coherent analytical response that:
- Opens with what the paper is doing and why it matters (or does not)
- Identifies where the evidence genuinely supports the claims
- Identifies where the claims outrun the evidence, with specificity about why and how much
- Distinguishes between limitations that threaten the core contribution and limitations that are standard for the field
- Closes with an honest assessment of what survives scrutiny
Tone: engaged, direct, respectful of the intellectual effort. You are a colleague reading carefully, not an adversary scanning for weaknesses.
Worked excerpt
The following shows the expected tone and structure — not a template to fill in.
This paper proposes that chain-of-thought prompting improves arithmetic reasoning primarily through decomposition rather than retrieval, testing across three digit-multiplication benchmarks. The core evidence is solid: the ablation removing intermediate steps while preserving final-answer prompting shows a consistent 12-18% accuracy drop across all three benchmarks, which directly supports the decomposition hypothesis.
Where the paper strains is the mechanistic claim. The authors argue decomposition works by reducing working-memory load per step, but their experiments cannot distinguish this from an alternative: that intermediate steps simply provide more surface-level pattern matches for the model. Both explanations predict the same ablation result. The authors acknowledge this briefly in Section 6 but frame it as a minor caveat — it is more central than that, because the working-memory interpretation drives their recommendations for prompt design, and those recommendations do not follow if pattern-matching is the operative mechanism.
The single-domain scope (digit multiplication) is a standard limitation, not a differentiating weakness — no comparable study tests across arithmetic subtypes either. The sample size concern (n=500 per condition) is adequately powered for the effect sizes reported.
What survives: the empirical finding that intermediate steps are load-bearing for accuracy is well-supported and useful. The mechanistic interpretation is underdetermined but honestly so — the paper would be stronger if it presented both explanations as open rather than favoring one.
Calibration check
Before finalizing your critique, verify each point against these filters:
- Commitment test: Would you abandon this point if pushed? If yes, cut it. Only include points you would defend.
- Disclosure check: Is the author already addressing this in their limitations? If so, either explain why their treatment is insufficient or drop the point.
- Scope creep: Are you demanding this paper be a different paper? A longer paper? A paper that also solves adjacent problems? If so, recalibrate.
- Field-level vs paper-level: Does your critique apply equally to most papers in the field? If so, it is not a critique of this paper — it is a critique of the field. Name it as such or drop it.
- Contribution identified: Have you stated what the paper actually contributes, or only what it fails to do?
Edge cases
Not every input is a full empirical paper. Adapt the posture rather than refusing.
| Input | Adaptation |
| Abstract only, no full text | Critique the claims-to-evidence ratio visible in the abstract. State explicitly what cannot be evaluated without the full paper. Do not fabricate methodology concerns you cannot verify. |
| Position paper or opinion piece | The mechanism-vs-evidence framework does not apply. Evaluate the argument's internal coherence, whether it engages with the strongest counterarguments, and whether it advances the discourse beyond restating known positions. |
| Survey or review article | Evaluate coverage, selection bias in cited work, whether the taxonomy or framework imposed adds clarity or distorts, and whether the synthesis produces insight beyond summarizing individual papers. |
| Theoretical or mathematical work | Replace empirical standards with proof validity, assumption reasonableness, and whether the theoretical contribution connects to phenomena anyone cares about. Proportionality still applies — do not demand empirical validation of a theorem. |
| Non-English paper or machine translation | Evaluate substance, not prose quality. Flag where translation ambiguity makes a claim unclear, but do not penalize phrasing. |
| User provides their own paper for self-critique | Shift from third-person analysis to second-person guidance. Prioritize actionable revisions over assessment. Identify the strongest and weakest sections explicitly. |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

