agentleFS
Sign inStart free
FmarzochiFmarzochi/EGC/skills/docs/scientific-thinking-scholar-evaluation/SKILL.md

scholar-evaluation

Skill61 starsChanged 46 days agoApache-2.0View on GitHub

Structured scholarly-work evaluation for papers, proposals, literature reviews, methods sections, evidence quality, citation support, and research-writing feedback.

We split this skill into the rules it gives an agent

13 rules. 0 are common, 12 are distinctive, and 1 is very specific to this project.

  • 0

    Common

    Most of the 16 edited versions we read keep these.

  • 12

    Distinctive

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this skill to evaluate academic or scientific work with a repeatable rubric.

    line 9
  2. Distinctive:

    Start by identifying the artifact:

    line 21
  3. Distinctive:

    comprehensive: all rubric dimensions

    line 33
  4. Very specific:

    Use N/A for dimensions that do not apply.

    line 47
  5. Distinctive:

    Do claims stay within the evidence?

    line 88
  6. Distinctive:

    Do cited papers support the claims attached to them?

    line 107
  7. Distinctive:

    Read the abstract, introduction, figures, and conclusion for claimed contribution.

    line 115
  8. Distinctive:

    Read methods and results for evidence quality.

    line 117
  9. Distinctive:

    Check the strongest claims against cited sources.

    line 118
  10. Distinctive:

    Do not use the score as a substitute for concrete feedback.

    line 157
  11. Distinctive:

    Do not penalize a paper for omitting a dimension outside its scope.

    line 158
  12. Distinctive:

    Do not treat citation count, venue, or author reputation as proof of quality.

    line 159
  13. Distinctive:

    Do not accept unsupported claims just because they appear in the abstract.

    line 160

“Common” is measured: kept by at least 6 in 10 of the 16 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Scholar Evaluation

Use this skill to evaluate academic or scientific work with a repeatable rubric.

When to Use

  • Reviewing a research paper, proposal, thesis chapter, or literature review.
  • Checking whether claims are supported by cited evidence.
  • Evaluating methodology, study design, analysis, or limitations.
  • Comparing two or more papers for quality or relevance.
  • Producing structured feedback for revision.

Evaluation Scope

Start by identifying the artifact:

  • empirical research paper
  • theoretical paper
  • technical report
  • systematic or narrative literature review
  • research proposal
  • thesis or dissertation chapter
  • conference abstract or short paper

Then choose scope:

  • comprehensive: all rubric dimensions
  • targeted: one or two dimensions, such as method or citations
  • comparative: rank multiple works against the same rubric

Rubric

Score each applicable dimension from 1 to 5:

  • 5: excellent; clear, rigorous, and publication-ready
  • 4: good; minor improvements needed
  • 3: adequate; meaningful gaps but usable
  • 2: weak; substantial revision needed
  • 1: poor; major validity or clarity problems

Use N/A for dimensions that do not apply.

1. Problem and Research Question

  • Is the problem clear and specific?
  • Is the contribution meaningful?
  • Are scope and assumptions explicit?
  • Does the question match the claimed contribution?

2. Literature and Context

  • Is relevant prior work covered?
  • Does the work synthesize rather than merely list sources?
  • Are gaps accurately identified?
  • Are recent and foundational sources balanced?

3. Methodology

  • Does the method answer the research question?
  • Are design choices justified?
  • Are variables, datasets, participants, or materials described clearly?
  • Could another researcher reproduce the work?
  • Are ethical and practical constraints acknowledged?

4. Data and Evidence

  • Are data sources credible and appropriate?
  • Is sample size or corpus coverage adequate?
  • Are inclusion, exclusion, and preprocessing decisions documented?
  • Are missing data and bias risks discussed?

5. Analysis

  • Are statistical, qualitative, or computational methods appropriate?
  • Are baselines and controls fair?
  • Are uncertainty, sensitivity, or robustness checks included when needed?
  • Are alternative explanations considered?

6. Results and Interpretation

  • Are results clearly presented?
  • Do claims stay within the evidence?
  • Are figures, tables, and metrics understandable?
  • Are negative or null results handled honestly?

7. Limitations and Threats to Validity

  • Are limitations specific rather than generic?
  • Are internal, external, construct, and conclusion-validity risks addressed?
  • Does the paper distinguish speculation from demonstrated results?

8. Writing and Structure

  • Is the argument easy to follow?
  • Are sections organized around the research question?
  • Are definitions and notation clear?
  • Is the tone precise and scholarly?

9. Citations

  • Do cited papers support the claims attached to them?
  • Are primary sources used where possible?
  • Are reviews labeled as reviews?
  • Are preprints labeled as preprints?
  • Are citation metadata and links correct?

Review Process

  1. Read the abstract, introduction, figures, and conclusion for claimed contribution.
  2. Read methods and results for evidence quality.
  3. Check the strongest claims against cited sources.
  4. Score each applicable dimension.
  5. Separate critical blockers from revision suggestions.
  6. End with concrete next edits.

Output Template

# Scholar Evaluation: <Artifact>

## Overall Assessment

- Overall score: <1-5 or N/A>
- Confidence: <high | medium | low>
- Summary: <3-5 sentences>

## Dimension Scores

| Dimension | Score | Evidence | Revision priority |
| --- | ---: | --- | --- |
| Problem and question |  |  |  |
| Literature and context |  |  |  |
| Methodology |  |  |  |
| Data and evidence |  |  |  |
| Analysis |  |  |  |
| Results and interpretation |  |  |  |
| Limitations |  |  |  |
| Writing and structure |  |  |  |
| Citations |  |  |  |

## Critical Issues

## Recommended Revisions

## Evidence Checks Needed

Pitfalls

  • Do not use the score as a substitute for concrete feedback.
  • Do not penalize a paper for omitting a dimension outside its scope.
  • Do not treat citation count, venue, or author reputation as proof of quality.
  • Do not accept unsupported claims just because they appear in the abstract.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

Reviews can't be read right now.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free