researcher-evaluator
Skill1 starsChanged 52 days agoMITView on GitHub
Stress-test a piece of research — a report, literature review, competitive analysis, or "I looked into it and..." summary — for rigor, bias, and hidden gaps before it gets relied on. Use whenever the user shares research (their own, another AI's, or a document) and asks whether it holds up, whether they missed something, or whether a conclusion is safe to act on. Give a blunt verdict, not a polite summary. Do not use this for grading writing style — this is about evidentiary rigor only.
In this repository32 other files it gives its agents.
- agent-system-designSkill
- api-contract-designSkill
- architecture-designSkill
- architecture-optimizationSkill
- ci-speedupSkill
- code-handoffSkill
- cross-ai-handoffSkill
- crypto-storage-auditSkill
- database-designSkill
- dataset-sourcingSkill
- distributed-debuggerSkill
- embedded-constrained-codingSkill
- freedom-to-operate-analysisSkill
- frontend-architectureSkill
- github-masterSkill
- idea-finderSkill
- ieee-paper-drafterSkill
- memory-leak-hunterSkill
- ml-model-selectionSkill
- ml-security-auditSkill
- mobile-platform-architectureSkill
- observability-designSkill
- patent-drafterSkill
- patent-novelty-evaluatorSkill
- performance-optimizationSkill
- project-scaffoldingSkill
- researcherSkill
- safety-compliance-scopingSkill
- scale-refactorSkill
- secure-code-reviewSkill
- slow-query-optimizerSkill
- test-architectSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 2 are distinctive, and 0 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
0
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 12
Read the full research output — every claim and every source — before forming a verdict.
- Distinctive: line 36
Never round a weak result up to "mostly fine" to soften delivery.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Researcher Evaluator
Research is easy to produce and easy to make look convincing. This skill's only job is to find where it's actually load-bearing versus where it's decoration, and say so without softening it.
Read the whole artifact first
Read the full research output — every claim and every source — before forming a verdict. Don't sample the executive summary and extrapolate.
Run this checklist against it
- Source quality — Is each claim backed by a primary source (paper, patent, official docs, company's own filing) or by an aggregator/blog/marketing page restating something else? Flag every load-bearing claim resting only on a secondary source.
- Recency — Does every claim about "current state" carry a date, and is that date actually recent enough to matter for a fast-moving domain? A 2022 claim about "the state of the art" in ML is often already wrong.
- Coverage, not just volume — Did the search actually cover the layer that could falsify the conclusion (patents, if novelty is claimed; shipped products, if "nothing like this exists" is claimed), or did it only search the layer that was easy and reassuring?
- Cherry-picking — Is there a search for disconfirming evidence at all, or does the report read like it stopped the moment it found support? Absence of counter-evidence in the report is not the same as absence of counter-evidence in the world — check whether the report distinguishes the two.
- Confidence calibration — Does the language's certainty match the actual evidence strength? "Definitely novel" backed by three quick searches is miscalibrated; flag every overclaim.
- Load-bearing claims, individually — Identify the 2-3 claims the final conclusion actually depends on. Each needs independent double-sourcing. If any of them has only one source, the conclusion is not yet safe to act on, regardless of how much other material surrounds it.
- Reproducibility — Could someone re-run the described searches and land on the same picture, or is the trail too vague to check (no query terms, no source list, just conclusions)?
Verdict
Give one of three verdicts, stated plainly:
- Solid — load-bearing claims are double-sourced, disconfirming search was actually attempted and came up empty, confidence matches evidence.
- Shaky — usable as a starting point, but named gaps must be closed before anyone acts on it.
- Trash — the conclusion isn't supported by what was actually checked; don't act on this as-is.
For every failure found, give:
- what's wrong (one line, specific — not "sourcing could be better")
- the exact fix — the specific search query, source type, or reference still missing
Never round a weak result up to "mostly fine" to soften delivery. The point of this skill is that the verdict is the same whether it's flattering or not.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

