assess-paper
Skill59 starsChanged 5 months agoApache-2.0View on GitHub
One consolidated read on how good a paper is right now — top strengths, biggest risks, scores, and what to do next — by running the compliance, integrity, citation, conformance, and mock-review skills and summarizing them in one Paper Health Report. Use when a researcher asks "how good is my paper", "is my paper ready", "what are the strengths and weaknesses of my paper", "give me an overall assessment", "review my whole paper", "what should I improve first", or "what's the state of my paper". Orchestrates preflight-check, verify-citations, check-originality, benchmark-paper, and simulate-reviewers, then summarizes them in plain language with strengths first, a prioritized fix list, and an honest readiness read — never a prediction of acceptance. Trigger words - how good is my paper, paper health, overall assessment, is it ready, strengths and weaknesses, what to improve, paper report, assess my paper.
In this repository43 other files it gives its agents.
- docs/llms-full.txtllms.txt
- docs/llms.txtllms.txt
- add-venue-profileSkill
- anonymize-paperSkill
- benchmark-paperSkill
- check-originalitySkill
- draft-related-workSkill
- draft-surveySkill
- fetch-paperSkill
- find-papersSkill
- fit-page-limitSkill
- literature-reviewSkill
- make-posterSkill
- make-slidesSkill
- match-styleSkill
- orchestrate-paperSkill
- paper-profileSkill
- parse-cfpSkill
- plan-submissionSkill
- polish-proseSkill
- polish-tables-figuresSkill
- preflight-checkSkill
- prepare-artifactsSkill
- prepare-camera-readySkill
- refactor-research-codeSkill
- refactor-structureSkill
- reflect-and-improveSkill
- reflect-paperSkill
- rehearse-qaSkill
- render-workspace-htmlSkill
- select-venueSkill
- simulate-reviewersSkill
- study-exemplarsSkill
- tailor-to-venueSkill
- test-research-codeSkill
- triage-reviewsSkill
- verify-citationsSkill
- verify-claimsSkill
- verify-resultsSkill
- work-with-overleafSkill
- write-abstractSkill
- write-rebuttalSkill
- write-talk-scriptSkill
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 0 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 107
At start: read profile.yml (positioning) and lessons.md (recurring weaknesses) so the report is personalized and tracks progress across …
- Very specific: line 109
Create .paper-memory/ on demand.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Assess Paper
The single command that tells an author where they stand and explains why, so a non-expert isn't left staring at five separate tool outputs. It does not re-implement any check; it runs the specialist skills and turns their results into one readable Paper Health Report that leads with strengths, then risks, then a ranked to-do list.
Pairs with orchestrate-paper (which drives the full lifecycle) — assess-paper is the read-only "where am I right now" snapshot you can run at any point. Copilot, not pilot: it reports and explains, it never edits the paper or submits anything.
When to use
- The author wants one overview instead of running five skills and stitching the results together.
- Early ("how far off am I?") or late ("am I ready to submit?").
- A co-author or advisor wants a quick, honest state-of-the-paper.
When NOT to use it (say this plainly)
- You want one specific check — run that skill directly (e.g. just citations →
verify-citations). - You want fixes applied — this skill only diagnoses; hand each fix to the skill that owns it.
- You want an acceptance probability or award forecast — no tool can give one honestly; this skill refuses to.
Inputs
- The draft: a
.texsource tree, a compiled PDF, or a readable draft. Processed transiently — never copied into this repo. - The target venue and track (what counts as "ready" differs by venue). Ask if unstated; run against the nearest family default and say so if no profile exists.
- Optional:
.paper-memory/profile.yml(positioning) andlessons.md(recurring weaknesses), if present.
What it consolidates
| Dimension | Skill it runs | What it contributes to the report |
| Will it get desk-rejected? | preflight-check | compliance blockers (page limit, anonymization, missing sections) |
| Are the citations real? | verify-citations | fabricated / retracted / duplicate references |
| Is it original? | check-originality | plagiarism / self-recycling overlap |
| Does it match strong work at the venue? | benchmark-paper | venue-fit scorecard + weakest dimensions |
| What will reviewers say? | simulate-reviewers | strengths, weaknesses, rubric scores, decision-risk |
Process
- Confirm the target (venue + track) and read
.paper-memory/profile.ymlif present, so the assessment matches the paper's positioning — a theory paper and an applied paper are judged differently. - Run the checks that apply to the draft's stage. Compliance and citations/originality are integrity gates; benchmark and mock-review are quality reads. Skip and say so if an input is missing (e.g. no
.bibyet) — never fabricate a result to fill a gap. - Synthesize the Paper Health Report (do not dump five raw outputs):
- Overall read — one honest paragraph: roughly where the paper stands and the single most important thing to do next. Never an acceptance probability.
- Top strengths (lead with these) — 3–5 concrete things the paper does well, drawn from the mock review and benchmark, each tied to a section, so the author knows what to protect while editing.
- Biggest risks — ranked by impact × ease of fix, each linking to the skill that fixes it.
- Scorecard + gates — the venue-fit index, mock-review decision-risk band, and a clear pass / flag on each integrity gate (citations, originality, desk-reject).
- Do next — a short ordered checklist.
- Write the report to
paper-workspace/review/paper-health-report.mdand present the summary in chat.
Any venue fact that flows through from the sub-skills (a page limit, a deadline, a required section) carries its source and confidence label exactly as the sub-skill reported it — do not restate a venue rule the underlying skill could not verify.
Worked mini-example
Input: a draft paper.tex + refs.bib, targeting sigspatial-2026 (full track). The synthesized report opens like this:
# Paper Health Report — paper.tex → SIGSPATIAL 2026 (full track) · 2026-06-21
## Overall read
Structurally this reads like a SIGSPATIAL full paper and the core method is
clearly framed. The one thing holding it back is the evaluation: a single
dataset with no significance test is the weakness every mock reviewer flagged.
Fix that before anything else.
## Top strengths (protect these)
- Crisp contribution list, 3 claims each mapped to a section (§1) — mock panel
praised the framing.
- Strong, reproducible artifact: code + seeds released (§6) — benchmark scored
reproducibility 9/10, above the exemplar median.
- Clear problem motivation tied to a real spatial workload (§2).
## Biggest risks (ranked by impact × ease)
1. Single-dataset evaluation, no significance test — add a second dataset +
variance. → simulate-reviewers (soundness), benchmark-paper (evaluation)
2. 2 references unresolvable on Crossref/DBLP (likely wrong year). → verify-citations
3. Page count at 9.3 / 9 — over the limit. → preflight-check, fit-page-limit
## Scorecard + gates
- Venue-fit index: 7.1/10 — "structurally in line; evaluation is the gap"
- Mock decision-risk: borderline-reject (no champion)
- Gates: citations FLAG (2 unresolvable) · originality PASS · desk-reject FLAG (over page limit)
## Do next
1. Resolve the 2 citations (verify-citations) — quick, removes an integrity flag.
2. Trim to the page limit (fit-page-limit).
3. Add the second dataset + significance test (the real lever).Output
A single paper-health-report.md: overall read → strengths → ranked risks → scorecard & gates → do-next checklist. Each finding names the underlying skill so the author can drill in. Plain language throughout — the goal is the author understanding their paper, not a wall of metrics.
Guardrails
- Strengths are mandatory and come first. An assessment that is all problems misleads the author about what to preserve — surface what genuinely works, but never invent strengths to pad.
- Honest readiness read only — never a probability of acceptance or an award (see
benchmark-paper,simulate-reviewers). Refuse if asked, and explain why. - Integrity gates (fabricated citations, plagiarism, desk-reject blockers) are reported as blockers, not averaged away by good scores elsewhere.
- It reports and explains; it never edits the paper or submits anything. Hand fixes to the specific skills.
- Process the paper transiently; never copy paper text into this repo.
Memory
Uses the shared .paper-memory/ convention described by paper-memory-convention.md.
- At start: read
profile.yml(positioning) andlessons.md(recurring weaknesses) so the report is personalized and tracks progress across drafts — lead with anyrecurringweakness. - At end: append durable takeaways in the shared format
- [YYYY-MM-DD] (assess-paper | <scope>) finding -> recommendationviareflect-and-improve'sreflect_log.py append(it dedupes and dates). A weakness that persists across drafts isrecurring; a one-time gap isthis-paper. Log lasting takeaways, not the full report or per-run scores. - Create
.paper-memory/on demand; it is local-only and never uploaded.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

