agentleFS
AMinerOrgAMinerOrg/aminer-research-skills/skills/pdf-citation-verifier/SKILL.md

pdf-citation-verifier

Skill81 starsChanged 7 days agoMITView on GitHub

[Activation] Use this skill when the user provides a paper PDF (file path or upload) and asks to verify, audit, or fact-check its references / citations / bibliography — e.g. "check whether the references in this PDF are hallucinated", "find fake citations", "verify the bibliography". [Capability] Uploads the PDF to the AMiner pdf-citation-verifier service, polls the asynchronous job, and returns a per-reference classification (REAL / LIKELY_REAL / NEEDS_REVIEW / LIKELY_FAKE / FAKE) plus an overall hallucination summary. [Routing] Do NOT use for general paper search, scholar lookup, citation-intent analysis, or building a citation graph — use aminer-academic-search, aminer-free-academic, or paper-source-trace instead. This skill only verifies whether references actually exist.

  • Reads credentials
  • Installs packages

The file

Show its rules and text

We split this skill into the rules it gives an agent

26 rules. About 0 are common, 6 are distinctive, and 20 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈6

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 20

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Verify whether the references in a paper PDF actually exist by submitting the PDF to the AMiner pdf-citation-verifier service, polling the …

    line 28
  2. Distinctive:

    Run these checks before invoking the script.

    line 58
  3. Very specific:

    If missing, stop and tell the user to obtain a token from https://open.aminer.cn and export AMINER_API_KEY=<token>.

    line 66
  4. Very specific:

    If missing, instruct: pip install -r "${CLAUDE_PLUGIN_ROOT}/requirements.txt".

    line 78
  5. Very specific:

    Treat any LIKELY_FAKE / FAKE verdict as a flag for human review, not a final accusation.

    line 146
  6. Distinctive:

    After the script returns, summarize the result for the user with at minimum:

    line 150
  7. Very specific:

    If details.records[] is present (auto-fetched from urls.result), list each FAKE / LIKELY_FAKE / NEEDS_REVIEW record's title, first_author, …

    line 156
  8. Very specific:

    If website_records[] is present, list each entry's raw (first ~80 chars), the reachable URL(s), and its original status.

    line 158
  9. Distinctive:

    After the verification finishes (and after the Local Non-Reference Filter when it applies), generate a visual report card with the standard …

    line 163
  10. Distinctive:

    All numbers come verbatim from the input JSON.

    line 174
  11. Very specific:

    Use only these record fields for the review: id, status, confidence, title, first_author, year, raw, key_reasons, and top_match.

    line 183
  12. Very specific:

    Add llm_filter only to records removed by the local review:

    line 188
  13. Very specific:

    Add filtered_records containing all retained records.

    line 192
  14. Very specific:

    Add removed_non_reference_records containing removed records with their llm_filter.

    line 193
  15. Very specific:

    Add filtered_summary recomputed only from filtered_records:

    line 194
  16. Very specific:

    If the inline details fetch failed, the payload carries a details_fetch_error string — surface it and tell the user to GET urls.result …

    line 202
  17. Very specific:

    If is_finish is true and a status / msg field signals failure, report it and suggest re-running.

    line 204
  18. Very specific:

    Consider only records whose status is one of LIKELY_REAL / NEEDS_REVIEW / LIKELY_FAKE / FAKE (skip REAL).

    line 213
  19. Very specific:

    Read URLs only from the raw field.

    line 214
  20. Very specific:

    If a record has no URL in raw, leave it untouched — no metadata added.

    line 215
  21. Very specific:

    Never fabricate an HTTP response.

    line 224
  22. Distinctive:

    Do not send record data to any third-party service beyond the URL host itself.

    line 225
  23. Distinctive:

    Add two new top-level fields to the returned payload:

    line 241
  24. Very specific:

    If the user gave --output path.json, also write path.website_verified.json alongside path.filtered.json.

    line 252
  25. Very specific:

    If no --output, do not create new files.

    line 253
  26. Very specific:

    Do NOT change the server-side hallucination_ratio or counts_by_status.

    line 255

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

PDF Citation Verifier

Verify whether the references in a paper PDF actually exist by submitting the PDF to the AMiner pdf-citation-verifier service, polling the asynchronous job, and returning a structured summary. Invoke via natural language or /pdf-citation-verifier.

What This Skill Does

For each reference parsed from the uploaded PDF, the upstream service queries AMiner SearchPro and labels the citation with one of:

  • REAL — high-confidence match in AMiner.
  • LIKELY_REAL — partial match, likely genuine.
  • NEEDS_REVIEW — evidence is inconclusive; ask a human.
  • LIKELY_FAKE — partial mismatch, probably fabricated.
  • FAKE — no plausible match found.

Each call to the gateway returns the standard envelope {"code": 200, "success": true, "msg": "", "data": ..., "log_id": "..."}. The script unwraps it before printing.

  • POST /api/v3/paper/citation/verify/upload returns data: {"job_id": "verify_..."}.
  • GET /api/v3/paper/citation/result?job_id=... returns data: [<record>] where the single record has top-level fields like is_finish, has_hallucination, hallucination_ratio, total, counts_by_status, summary, urls, report, result.
  • Whenever the script sees is_finish: true, it also auto-downloads urls.result (the per-reference JSON) and inlines it as details on the returned payload — so a single --output file contains both the summary and every record's status / confidence / title / first_author / key_reasons / top_match, with no need to follow the 5-minute OSS link.

The skill returns that record plus the job_id so the user can re-poll later.

File Map

  • SKILL.md / SKILL.zh.md — English / Chinese skill definitions (this file).
  • commands/pdf-citation-verifier.md — slash command entry.
  • scripts/verify_pdf.py — HTTP client: upload → poll → print the unwrapped result record.
  • scripts/render_report.py — renders the result JSON into a self-contained HTML report card (stdlib only).
  • requirements.txt — Python dependencies (requests).

Pre-flight

Run these checks before invoking the script. Stop and surface the error to the user if any check fails.

1. AMINER_API_KEY

[ -z "${AMINER_API_KEY+x}" ] && echo "AMINER_API_KEY missing" || echo "AMINER_API_KEY exists"

If missing, stop and tell the user to obtain a token from https://open.aminer.cn and export AMINER_API_KEY=<token>. Never print the token value.

2. Python dependency

python3 - <<'PY'
import importlib.util
missing = [name for name in ("requests",) if importlib.util.find_spec(name) is None]
print("Missing: " + ", ".join(missing) if missing else "Python dependencies exist")
PY

If missing, instruct: pip install -r "${CLAUDE_PLUGIN_ROOT}/requirements.txt".

3. PDF input

The user must supply an existing local .pdf file path. If they only describe a paper without a file, ask them to provide the PDF path. Do not invent or download a PDF.

Execution Example

Basic verification with defaults (max 50 references, auto-polls until done):

python3 "${CLAUDE_PLUGIN_ROOT}/scripts/verify_pdf.py" \
  --pdf "/abs/path/to/paper.pdf"

Full options:

python3 "${CLAUDE_PLUGIN_ROOT}/scripts/verify_pdf.py" \
  --pdf "/abs/path/to/paper.pdf" \
  --max-refs 80 \
  --strict \
  --timeout 900 \
  --poll-interval 5 \
  --output outputs/pdf-citation-verifier/<safe-paper-stem>/result.json

Submit-only (no polling, return job_id for later lookup):

python3 "${CLAUDE_PLUGIN_ROOT}/scripts/verify_pdf.py" \
  --pdf "/abs/path/to/paper.pdf" --no-wait

Fetch the result for an existing job_id (no new upload):

python3 "${CLAUDE_PLUGIN_ROOT}/scripts/verify_pdf.py" \
  --job-id verify_20260527T090207Z_a72c9ba5

Parameters

FlagDefaultNotes
--pdfrequired (unless --job-id)Local .pdf file path. Server caps body at 50 MB.
--job-id–Skip upload and just fetch the result for an existing job.
--max-refs50Server hard cap is 100.
--strictoffStricter FAKE judgement on partial matches.
--no-waitoffWith --pdf: submit and return job_id without polling. With --job-id: single fetch, return immediately without looping.
--timeout600Overall polling timeout in seconds.
--poll-interval5Seconds between result polls.
--request-timeout120Per-HTTP-request timeout.
--output-Optional path to also write the JSON response.

Environment Variables

VarRequiredPurpose
AMINER_API_KEYyesJWT used in the Authorization header.
PDF_CITATION_VERIFIER_BASE_URLnoOverride the gateway base URL. Defaults to https://datacenter.aminer.cn/gateway/open_platform.

Runtime Constraints

  • Never print, log, or echo the value of AMINER_API_KEY.
  • Never fabricate verification verdicts. If the script fails or times out, surface the error verbatim — do not synthesize results.
  • urls, report, result, pdf in the response point to server-side artifacts and may be pre-signed for url_expire_seconds. Do not claim those local paths exist on the user's machine. Use --output if the user needs a local copy of the JSON.
  • Respect the per-user active job cap (server returns 429 when exceeded). If a 429 surfaces, stop and tell the user to wait for prior jobs to finish.
  • Treat any LIKELY_FAKE / FAKE verdict as a flag for human review, not a final accusation. Surface counts_by_status and per-record reasons when the response includes them.

Output Presentation

After the script returns, summarize the result for the user with at minimum:

  • job_id
  • total (number of references verified)
  • has_hallucination, hallucination_ratio
  • A short table built from counts_by_status (REAL / LIKELY_REAL / NEEDS_REVIEW / LIKELY_FAKE / FAKE / etc.)
  • If details.records[] is present (auto-fetched from urls.result), list each FAKE / LIKELY_FAKE / NEEDS_REVIEW record's title, first_author, year, and key_reasons so the user does not have to follow the 5-minute OSS link
  • Any urls.report / urls.result links from the response, with a note that they may expire after url_expire_seconds
  • If website_records[] is present, list each entry's raw (first ~80 chars), the reachable URL(s), and its original status; then show website_summary (total_website_citations, adjusted_hallucination_ratio). Explicitly note that adjusted_hallucination_ratio excludes website-type citations and that the original hallucination_ratio is still in the payload.
  • The full JSON should be either saved (via --output) or echoed back to the user, never silently dropped.

HTML Report Card

After the verification finishes (and after the Local Non-Reference Filter when it applies), generate a visual report card with the standard renderer — do not hand-write ad hoc HTML:

python3 "${CLAUDE_PLUGIN_ROOT}/scripts/render_report.py" \
  --input "<result-json-path>" \
  --output "<same-dir>/report.html" \
  --lang zh   # zh when the conversation is mainly Chinese, en otherwise
  • --input is the JSON written by verify_pdf.py --output (must contain summary; details.records enables the per-record focus table). Prefer the .filtered.json when the Local Non-Reference Filter produced one.
  • The output is a single self-contained HTML file (inline CSS, no external assets): an overview card with the fake-ratio headline number, a five-color stacked status bar with legend, and a focus table listing each FAKE / LIKELY_FAKE / NEEDS_REVIEW record with its verdict badge, citation info, closest top_match, and human-readable reasons.
  • All numbers come verbatim from the input JSON; the renderer never re-classifies records.
  • Tell the user the report path and offer to open it in the browser. If no --output JSON exists (chat-only run), skip the report or first write the payload to a temp JSON.

Local Non-Reference Filter

When is_finish=true and details.records[] is present, perform a local agent review before presenting the final summary. This review uses the current Claude Code/Codex agent only; do not call another LLM API, do not ask for an extra key, and do not send records back to the server.

Filter conservatively. Remove only records that are clearly not bibliography references, such as body-text contamination, figure/table/stage/evaluation prose, appendix or section headings, page headers/footers, page numbers, captions, or non-citation narrative sentences. Do not remove a record merely because its status is FAKE, LIKELY_FAKE, or NEEDS_REVIEW, because AMiner has no match, or because metadata is incomplete. Ambiguous records stay in filtered_records.

Use only these record fields for the review: id, status, confidence, title, first_author, year, raw, key_reasons, and top_match. Prioritize raw; use title and key_reasons only as supporting signals.

Produce an auditable filtered payload:

  • Preserve the original service payload unchanged.
  • Add llm_filter only to records removed by the local review:
    • is_reference: false
    • reason: "<short reason>"
    • confidence: <0.0-1.0>
  • Add filtered_records containing all retained records.
  • Add removed_non_reference_records containing removed records with their llm_filter.
  • Add filtered_summary recomputed only from filtered_records:
    • total
    • counts_by_status
    • has_hallucination
    • hallucination_ratio = (FAKE + LIKELY_FAKE) / total, or 0 when total is 0

If the user supplied --output path.json, write the filtered payload next to it as path.filtered.json (for example, result.json -> result.filtered.json). If the output path has no .json suffix, append .filtered.json. If no --output was used, do not create a new file; present the filtered summary and removed records in the final answer.

If the inline details fetch failed, the payload carries a details_fetch_error string — surface it and tell the user to GET urls.result themselves before url_expire_seconds runs out.

If is_finish is true and a status / msg field signals failure, report it and suggest re-running.

URL / Website Citation Re-verification

After the Non-Reference filter completes and filtered_records is ready, run a second pass to identify website-type citations — references that point to a real webpage (RFC, blog post, GitHub README, standards doc, product page) rather than a journal / conference paper. AMiner SearchPro is paper-centric, so those citations often land in LIKELY_REAL / NEEDS_REVIEW / LIKELY_FAKE / FAKE even when the underlying URL is perfectly valid. This pass flags them so the user can distinguish "fake academic reference" from "legitimate website citation".

Scope:

  • Operate only on filtered_records (never on removed_non_reference_records).
  • Consider only records whose status is one of LIKELY_REAL / NEEDS_REVIEW / LIKELY_FAKE / FAKE (skip REAL).
  • Read URLs only from the raw field. Extract with a conservative regex like https?://[^\s<>"'()]+. Deduplicate per record.
  • If a record has no URL in raw, leave it untouched — no metadata added.

Probing rules (use the hosting agent's WebFetch tool; do NOT call any external LLM API and do NOT require an extra key):

  • HTTP 2xx or 3xx (allow up to 3 redirects) → reachable.
  • Any other status / timeout / DNS failure → unreachable.
  • 401 / 403 → unreachable_paywalled (do not treat as reachable).
  • Skip URLs that resolve to localhost or private IP ranges (10/8, 172.16/12, 192.168/16, 169.254/16, ::1, fc00::/7).
  • Per-URL timeout: 15s. At most 3 URLs per record; if a record has more, set url_probe_truncated: true on that record and probe the first 3 only.
  • Never fabricate an HTTP response. On WebFetch failure, record unreachable with the error verbatim in reason.
  • Do not send record data to any third-party service beyond the URL host itself.

For every record that contains at least one URL, attach a url_verification object:

"url_verification": {
  "urls": ["<url1>", "<url2>"],
  "reachable_urls": ["<url1>"],
  "is_website_citation": true,
  "reason": "<short reason>",
  "url_probe_truncated": false
}

is_website_citation is true iff at least one URL is reachable.

Add two new top-level fields to the returned payload:

  • website_records: every record where is_website_citation: true. Do NOT change its original status. The WEBSITE label lives only in the agent-side view; the server verdict is preserved for audit.
  • website_summary:
    • total_website_citations: N (count of records in website_records)
    • from_status: { "LIKELY_REAL": a, "NEEDS_REVIEW": b, "LIKELY_FAKE": c, "FAKE": d }
    • adjusted_hallucination_ratio: (FAKE_count + LIKELY_FAKE_count − c − d) / max(filtered_total − N, 1)
    • note: "WEBSITE citations excluded from academic-hallucination ratio."

Output file conventions (mirror the Non-Reference filter):

  • If the user gave --output path.json, also write path.website_verified.json alongside path.filtered.json. If the output path has no .json suffix, append .website_verified.json.
  • If no --output, do not create new files; only surface the results in the reply.

Do NOT change the server-side hallucination_ratio or counts_by_status. All website-aware numbers live in website_summary only.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.