pdf-citation-verifier
Skill81 starsChanged 7 days agoMITView on GitHub
[Activation] Use this skill when the user provides a paper PDF (file path or upload) and asks to verify, audit, or fact-check its references / citations / bibliography — e.g. "check whether the references in this PDF are hallucinated", "find fake citations", "verify the bibliography". [Capability] Uploads the PDF to the AMiner pdf-citation-verifier service, polls the asynchronous job, and returns a per-reference classification (REAL / LIKELY_REAL / NEEDS_REVIEW / LIKELY_FAKE / FAKE) plus an overall hallucination summary. [Routing] Do NOT use for general paper search, scholar lookup, citation-intent analysis, or building a citation graph — use aminer-academic-search, aminer-free-academic, or paper-source-trace instead. This skill only verifies whether references actually exist.
- Reads credentials
- Installs packages
In this repository21 other files it gives its agents.
- aminer-academic-searchSkill
- aminer-daily-paperSkill
- aminer-deep-searchSkill
- aminer-experiment-to-paperSkill
- aminer-exp-extractionSkill
- aminer-free-academicSkill
- aminer-grant-applicationSkill
- aminer-paper-deep-readingSkill
- aminer-paper-writingSkill
- aminer-patent-tech-intelligenceSkill
- aminer-pdf-ocrSkill
- aminer-thesis-writingSkill
- aminer-topic-advisor-selectionSkill
- baseline-finderSkill
- citation-faithfulnessSkill
- deep-researchSkill
- paper-source-traceSkill
- related-work-auditSkill
- scientific-taxonomySkill
- sota-finderSkill
- structured-reference-auditSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
26 rules. About 0 are common, 6 are distinctive, and 20 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈6
Distinctive · estimate
This author's own choices. Worth a look.
20
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 28
Verify whether the references in a paper PDF actually exist by submitting the PDF to the AMiner pdf-citation-verifier service, polling the …
- Distinctive: line 58
Run these checks before invoking the script.
- Very specific: line 66
If missing, stop and tell the user to obtain a token from https://open.aminer.cn and export AMINER_API_KEY=<token>.
- Very specific: line 78
If missing, instruct: pip install -r "${CLAUDE_PLUGIN_ROOT}/requirements.txt".
- Very specific: line 146
Treat any LIKELY_FAKE / FAKE verdict as a flag for human review, not a final accusation.
- Distinctive: line 150
After the script returns, summarize the result for the user with at minimum:
- Very specific: line 156
If details.records[] is present (auto-fetched from urls.result), list each FAKE / LIKELY_FAKE / NEEDS_REVIEW record's title, first_author, …
- Very specific: line 158
If website_records[] is present, list each entry's raw (first ~80 chars), the reachable URL(s), and its original status.
- Distinctive: line 163
After the verification finishes (and after the Local Non-Reference Filter when it applies), generate a visual report card with the standard …
- Distinctive: line 174
All numbers come verbatim from the input JSON.
- Very specific: line 183
Use only these record fields for the review: id, status, confidence, title, first_author, year, raw, key_reasons, and top_match.
- Very specific: line 188
Add llm_filter only to records removed by the local review:
- Very specific: line 192
Add filtered_records containing all retained records.
- Very specific: line 193
Add removed_non_reference_records containing removed records with their llm_filter.
- Very specific: line 194
Add filtered_summary recomputed only from filtered_records:
- Very specific: line 202
If the inline details fetch failed, the payload carries a details_fetch_error string — surface it and tell the user to GET urls.result …
- Very specific: line 204
If is_finish is true and a status / msg field signals failure, report it and suggest re-running.
- Very specific: line 213
Consider only records whose status is one of LIKELY_REAL / NEEDS_REVIEW / LIKELY_FAKE / FAKE (skip REAL).
- Very specific: line 214
Read URLs only from the raw field.
- Very specific: line 215
If a record has no URL in raw, leave it untouched — no metadata added.
- Very specific: line 224
Never fabricate an HTTP response.
- Distinctive: line 225
Do not send record data to any third-party service beyond the URL host itself.
- Distinctive: line 241
Add two new top-level fields to the returned payload:
- Very specific: line 252
If the user gave --output path.json, also write path.website_verified.json alongside path.filtered.json.
- Very specific: line 253
If no --output, do not create new files.
- Very specific: line 255
Do NOT change the server-side hallucination_ratio or counts_by_status.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
PDF Citation Verifier
Verify whether the references in a paper PDF actually exist by submitting the PDF to the AMiner pdf-citation-verifier service, polling the asynchronous job, and returning a structured summary. Invoke via natural language or /pdf-citation-verifier.
What This Skill Does
For each reference parsed from the uploaded PDF, the upstream service queries AMiner SearchPro and labels the citation with one of:
REAL— high-confidence match in AMiner.LIKELY_REAL— partial match, likely genuine.NEEDS_REVIEW— evidence is inconclusive; ask a human.LIKELY_FAKE— partial mismatch, probably fabricated.FAKE— no plausible match found.
Each call to the gateway returns the standard envelope {"code": 200, "success": true, "msg": "", "data": ..., "log_id": "..."}. The script unwraps it before printing.
POST /api/v3/paper/citation/verify/uploadreturnsdata: {"job_id": "verify_..."}.GET /api/v3/paper/citation/result?job_id=...returnsdata: [<record>]where the single record has top-level fields likeis_finish,has_hallucination,hallucination_ratio,total,counts_by_status,summary,urls,report,result.- Whenever the script sees
is_finish: true, it also auto-downloadsurls.result(the per-reference JSON) and inlines it asdetailson the returned payload — so a single--outputfile contains both the summary and every record'sstatus/confidence/title/first_author/key_reasons/top_match, with no need to follow the 5-minute OSS link.
The skill returns that record plus the job_id so the user can re-poll later.
File Map
SKILL.md/SKILL.zh.md— English / Chinese skill definitions (this file).commands/pdf-citation-verifier.md— slash command entry.scripts/verify_pdf.py— HTTP client: upload → poll → print the unwrapped result record.scripts/render_report.py— renders the result JSON into a self-contained HTML report card (stdlib only).requirements.txt— Python dependencies (requests).
Pre-flight
Run these checks before invoking the script. Stop and surface the error to the user if any check fails.
1. AMINER_API_KEY
[ -z "${AMINER_API_KEY+x}" ] && echo "AMINER_API_KEY missing" || echo "AMINER_API_KEY exists"If missing, stop and tell the user to obtain a token from https://open.aminer.cn and export AMINER_API_KEY=<token>. Never print the token value.
2. Python dependency
python3 - <<'PY'
import importlib.util
missing = [name for name in ("requests",) if importlib.util.find_spec(name) is None]
print("Missing: " + ", ".join(missing) if missing else "Python dependencies exist")
PYIf missing, instruct: pip install -r "${CLAUDE_PLUGIN_ROOT}/requirements.txt".
3. PDF input
The user must supply an existing local .pdf file path. If they only describe a paper without a file, ask them to provide the PDF path. Do not invent or download a PDF.
Execution Example
Basic verification with defaults (max 50 references, auto-polls until done):
python3 "${CLAUDE_PLUGIN_ROOT}/scripts/verify_pdf.py" \
--pdf "/abs/path/to/paper.pdf"Full options:
python3 "${CLAUDE_PLUGIN_ROOT}/scripts/verify_pdf.py" \
--pdf "/abs/path/to/paper.pdf" \
--max-refs 80 \
--strict \
--timeout 900 \
--poll-interval 5 \
--output outputs/pdf-citation-verifier/<safe-paper-stem>/result.jsonSubmit-only (no polling, return job_id for later lookup):
python3 "${CLAUDE_PLUGIN_ROOT}/scripts/verify_pdf.py" \
--pdf "/abs/path/to/paper.pdf" --no-waitFetch the result for an existing job_id (no new upload):
python3 "${CLAUDE_PLUGIN_ROOT}/scripts/verify_pdf.py" \
--job-id verify_20260527T090207Z_a72c9ba5Parameters
| Flag | Default | Notes |
--pdf | required (unless --job-id) | Local .pdf file path. Server caps body at 50 MB. |
--job-id | – | Skip upload and just fetch the result for an existing job. |
--max-refs | 50 | Server hard cap is 100. |
--strict | off | Stricter FAKE judgement on partial matches. |
--no-wait | off | With --pdf: submit and return job_id without polling. With --job-id: single fetch, return immediately without looping. |
--timeout | 600 | Overall polling timeout in seconds. |
--poll-interval | 5 | Seconds between result polls. |
--request-timeout | 120 | Per-HTTP-request timeout. |
--output | - | Optional path to also write the JSON response. |
Environment Variables
| Var | Required | Purpose |
AMINER_API_KEY | yes | JWT used in the Authorization header. |
PDF_CITATION_VERIFIER_BASE_URL | no | Override the gateway base URL. Defaults to https://datacenter.aminer.cn/gateway/open_platform. |
Runtime Constraints
- Never print, log, or echo the value of
AMINER_API_KEY. - Never fabricate verification verdicts. If the script fails or times out, surface the error verbatim — do not synthesize results.
urls,report,result,pdfin the response point to server-side artifacts and may be pre-signed forurl_expire_seconds. Do not claim those local paths exist on the user's machine. Use--outputif the user needs a local copy of the JSON.- Respect the per-user active job cap (server returns 429 when exceeded). If a 429 surfaces, stop and tell the user to wait for prior jobs to finish.
- Treat any
LIKELY_FAKE/FAKEverdict as a flag for human review, not a final accusation. Surfacecounts_by_statusand per-record reasons when the response includes them.
Output Presentation
After the script returns, summarize the result for the user with at minimum:
job_idtotal(number of references verified)has_hallucination,hallucination_ratio- A short table built from
counts_by_status(REAL / LIKELY_REAL / NEEDS_REVIEW / LIKELY_FAKE / FAKE / etc.) - If
details.records[]is present (auto-fetched fromurls.result), list each FAKE / LIKELY_FAKE / NEEDS_REVIEW record'stitle,first_author,year, andkey_reasonsso the user does not have to follow the 5-minute OSS link - Any
urls.report/urls.resultlinks from the response, with a note that they may expire afterurl_expire_seconds - If
website_records[]is present, list each entry'sraw(first ~80 chars), the reachable URL(s), and its originalstatus; then showwebsite_summary(total_website_citations,adjusted_hallucination_ratio). Explicitly note thatadjusted_hallucination_ratioexcludes website-type citations and that the originalhallucination_ratiois still in the payload. - The full JSON should be either saved (via
--output) or echoed back to the user, never silently dropped.
HTML Report Card
After the verification finishes (and after the Local Non-Reference Filter when it applies), generate a visual report card with the standard renderer — do not hand-write ad hoc HTML:
python3 "${CLAUDE_PLUGIN_ROOT}/scripts/render_report.py" \
--input "<result-json-path>" \
--output "<same-dir>/report.html" \
--lang zh # zh when the conversation is mainly Chinese, en otherwise--inputis the JSON written byverify_pdf.py --output(must containsummary;details.recordsenables the per-record focus table). Prefer the.filtered.jsonwhen the Local Non-Reference Filter produced one.- The output is a single self-contained HTML file (inline CSS, no external assets): an overview card with the fake-ratio headline number, a five-color stacked status bar with legend, and a focus table listing each FAKE / LIKELY_FAKE / NEEDS_REVIEW record with its verdict badge, citation info, closest
top_match, and human-readable reasons. - All numbers come verbatim from the input JSON; the renderer never re-classifies records.
- Tell the user the report path and offer to
openit in the browser. If no--outputJSON exists (chat-only run), skip the report or first write the payload to a temp JSON.
Local Non-Reference Filter
When is_finish=true and details.records[] is present, perform a local agent review before presenting the final summary. This review uses the current Claude Code/Codex agent only; do not call another LLM API, do not ask for an extra key, and do not send records back to the server.
Filter conservatively. Remove only records that are clearly not bibliography references, such as body-text contamination, figure/table/stage/evaluation prose, appendix or section headings, page headers/footers, page numbers, captions, or non-citation narrative sentences. Do not remove a record merely because its status is FAKE, LIKELY_FAKE, or NEEDS_REVIEW, because AMiner has no match, or because metadata is incomplete. Ambiguous records stay in filtered_records.
Use only these record fields for the review: id, status, confidence, title, first_author, year, raw, key_reasons, and top_match. Prioritize raw; use title and key_reasons only as supporting signals.
Produce an auditable filtered payload:
- Preserve the original service payload unchanged.
- Add
llm_filteronly to records removed by the local review:is_reference: falsereason: "<short reason>"confidence: <0.0-1.0>
- Add
filtered_recordscontaining all retained records. - Add
removed_non_reference_recordscontaining removed records with theirllm_filter. - Add
filtered_summaryrecomputed only fromfiltered_records:totalcounts_by_statushas_hallucinationhallucination_ratio = (FAKE + LIKELY_FAKE) / total, or0when total is0
If the user supplied --output path.json, write the filtered payload next to it as path.filtered.json (for example, result.json -> result.filtered.json). If the output path has no .json suffix, append .filtered.json. If no --output was used, do not create a new file; present the filtered summary and removed records in the final answer.
If the inline details fetch failed, the payload carries a details_fetch_error string — surface it and tell the user to GET urls.result themselves before url_expire_seconds runs out.
If is_finish is true and a status / msg field signals failure, report it and suggest re-running.
URL / Website Citation Re-verification
After the Non-Reference filter completes and filtered_records is ready, run a second pass to identify website-type citations — references that point to a real webpage (RFC, blog post, GitHub README, standards doc, product page) rather than a journal / conference paper. AMiner SearchPro is paper-centric, so those citations often land in LIKELY_REAL / NEEDS_REVIEW / LIKELY_FAKE / FAKE even when the underlying URL is perfectly valid. This pass flags them so the user can distinguish "fake academic reference" from "legitimate website citation".
Scope:
- Operate only on
filtered_records(never onremoved_non_reference_records). - Consider only records whose
statusis one ofLIKELY_REAL / NEEDS_REVIEW / LIKELY_FAKE / FAKE(skipREAL). - Read URLs only from the
rawfield. Extract with a conservative regex likehttps?://[^\s<>"'()]+. Deduplicate per record. - If a record has no URL in
raw, leave it untouched — no metadata added.
Probing rules (use the hosting agent's WebFetch tool; do NOT call any external LLM API and do NOT require an extra key):
- HTTP 2xx or 3xx (allow up to 3 redirects) →
reachable. - Any other status / timeout / DNS failure →
unreachable. - 401 / 403 →
unreachable_paywalled(do not treat as reachable). - Skip URLs that resolve to
localhostor private IP ranges (10/8, 172.16/12, 192.168/16, 169.254/16, ::1, fc00::/7). - Per-URL timeout: 15s. At most 3 URLs per record; if a record has more, set
url_probe_truncated: trueon that record and probe the first 3 only. - Never fabricate an HTTP response. On WebFetch failure, record
unreachablewith the error verbatim inreason. - Do not send record data to any third-party service beyond the URL host itself.
For every record that contains at least one URL, attach a url_verification object:
"url_verification": {
"urls": ["<url1>", "<url2>"],
"reachable_urls": ["<url1>"],
"is_website_citation": true,
"reason": "<short reason>",
"url_probe_truncated": false
}is_website_citation is true iff at least one URL is reachable.
Add two new top-level fields to the returned payload:
website_records: every record whereis_website_citation: true. Do NOT change its originalstatus. The WEBSITE label lives only in the agent-side view; the server verdict is preserved for audit.website_summary:total_website_citations: N (count of records inwebsite_records)from_status:{ "LIKELY_REAL": a, "NEEDS_REVIEW": b, "LIKELY_FAKE": c, "FAKE": d }adjusted_hallucination_ratio:(FAKE_count + LIKELY_FAKE_count − c − d) / max(filtered_total − N, 1)note:"WEBSITE citations excluded from academic-hallucination ratio."
Output file conventions (mirror the Non-Reference filter):
- If the user gave
--output path.json, also writepath.website_verified.jsonalongsidepath.filtered.json. If the output path has no.jsonsuffix, append.website_verified.json. - If no
--output, do not create new files; only surface the results in the reply.
Do NOT change the server-side hallucination_ratio or counts_by_status. All website-aware numbers live in website_summary only.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

