agentleFS
hemapriyan-rkhemapriyan-rk/claude-rigor-skills/skills/researcher-evaluator/SKILL.md

researcher-evaluator

Skill1 starsChanged 52 days agoMITView on GitHub

Stress-test a piece of research — a report, literature review, competitive analysis, or "I looked into it and..." summary — for rigor, bias, and hidden gaps before it gets relied on. Use whenever the user shares research (their own, another AI's, or a document) and asks whether it holds up, whether they missed something, or whether a conclusion is safe to act on. Give a blunt verdict, not a polite summary. Do not use this for grading writing style — this is about evidentiary rigor only.

The file

We split this skill into the rules it gives an agent

2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈2

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Read the full research output — every claim and every source — before forming a verdict.

    line 12
  2. Distinctive:

    Never round a weak result up to "mostly fine" to soften delivery.

    line 36

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Researcher Evaluator

Research is easy to produce and easy to make look convincing. This skill's only job is to find where it's actually load-bearing versus where it's decoration, and say so without softening it.

Read the whole artifact first

Read the full research output — every claim and every source — before forming a verdict. Don't sample the executive summary and extrapolate.

Run this checklist against it

  1. Source quality — Is each claim backed by a primary source (paper, patent, official docs, company's own filing) or by an aggregator/blog/marketing page restating something else? Flag every load-bearing claim resting only on a secondary source.
  2. Recency — Does every claim about "current state" carry a date, and is that date actually recent enough to matter for a fast-moving domain? A 2022 claim about "the state of the art" in ML is often already wrong.
  3. Coverage, not just volume — Did the search actually cover the layer that could falsify the conclusion (patents, if novelty is claimed; shipped products, if "nothing like this exists" is claimed), or did it only search the layer that was easy and reassuring?
  4. Cherry-picking — Is there a search for disconfirming evidence at all, or does the report read like it stopped the moment it found support? Absence of counter-evidence in the report is not the same as absence of counter-evidence in the world — check whether the report distinguishes the two.
  5. Confidence calibration — Does the language's certainty match the actual evidence strength? "Definitely novel" backed by three quick searches is miscalibrated; flag every overclaim.
  6. Load-bearing claims, individually — Identify the 2-3 claims the final conclusion actually depends on. Each needs independent double-sourcing. If any of them has only one source, the conclusion is not yet safe to act on, regardless of how much other material surrounds it.
  7. Reproducibility — Could someone re-run the described searches and land on the same picture, or is the trail too vague to check (no query terms, no source list, just conclusions)?

Verdict

Give one of three verdicts, stated plainly:

  • Solid — load-bearing claims are double-sourced, disconfirming search was actually attempted and came up empty, confidence matches evidence.
  • Shaky — usable as a starting point, but named gaps must be closed before anyone acts on it.
  • Trash — the conclusion isn't supported by what was actually checked; don't act on this as-is.

For every failure found, give:

  • what's wrong (one line, specific — not "sourcing could be better")
  • the exact fix — the specific search query, source type, or reference still missing

Never round a weak result up to "mostly fine" to soften delivery. The point of this skill is that the verdict is the same whether it's flattering or not.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.