citation-crosscheck
Skill18k starsChanged 4 months agoMITView on GitHub
Conditionally double-blind bibliographic cross-check (double-blind only when the host can isolate the web agent; a flag-only heuristic when isolation can't be attested): one agent extracts each cited entry verbatim from the local .bib, a second independently re-fetches the same paper's record from the web while blind to the .bib, and a third extractor emits a mechanical field-by-field delta to catch dropped/re-ordered authors, wrong years, wrong venues, and bad DOIs/arXiv ids. The main agent (never a shard) assigns the MATCH/MINOR/MISMATCH verdict per reference, and exports a side-by-side spreadsheet. Use when user says \"citation crosscheck\", \"double-blind citation check\", \"verify bib against scholar\", \"bib field audit\", \"引用交叉核对\", or before submission when bibliography FIELD accuracy (not just existence) must be certified.
We split this skill into the rules it gives an agent
6 rules. About 0 are common, 0 are distinctive, and 6 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
6
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 304
Agent A (repo-only): read the .bib; extract each cited entry verbatim (brace-matched, no reformatting) → /tmp/cx_A_ourbib.json …
- Very specific: line 323
If a title is not on arXiv, B MUST also run a general web search and open the landing page its own search returns (a project / blog / …
- Very specific: line 460
No verdict arrives pre-assigned from a shard.
- Very specific: line 464
No shard signal — a differs, an absent, B's found:false, or a routing flag — is itself a finding.
- Very specific: line 522
Add a Summary sheet with the counts and the fixed/optional lists.
- Very specific: line 529
Blindness is the mechanism: If B ever sees the .bib, the cross-check collapses into a single-source read and loses its power.
≈ “Common” is an estimate: at least 1 in 5 of the 308,232 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Citation Cross-Check (conditionally double-blind bib-vs-web field audit)
Codex mirror adaptation (normative). This skill has no cross-model jury by design — it is an evidence instrument, not an acquittal step (see the acceptance-gate note in Step 2, and pair it with
citation-auditfor the cross-model verdict). A and B are read-only extraction shards and C emits a mechanical per-field delta, so every verdict is assigned by the main agent from that delta plus its own primary-source fetch — not by any shard and not by a reviewer model.Blindness is NOT automatically identical across hosts — it is conditional on isolation, and the gate enforces that. The whole cross-check rests on B never seeing the
.bib. On the mainline, theAgenttool gives B a fresh context. On a Codex host the frontmatter grants no delegation tool (noAgent, nospawn_agent— seeallowed-tools), so:
- If the host can dispatch B in a genuinely isolated context that does not inherit the main agent's history (which has already read the
.bibin Step 1) — passing B only the title+first-author list — B may setblind:trueand the run certifies normally.- Otherwise (B would run in the main context, or "fresh sequential context" cannot be guaranteed to drop the parent's
.bibview), B MUST setblind:false. The Step-3.5 gate then downgrades everyMATCH/MINORtoESCALATE: the run still flags discrepancies for the main agent's Step-4 webcheck, but it certifies nothing. It is a labelled heuristic, not a double-blind certification.So the mirror never claims blindness it cannot deliver: it either has real isolation and attests it, or it declares
blind:falseand drops to flag-only. Do not add aspawn_agent-based reviewer/continuation route — the trust decision is the main agent'sWebFetch, and it must not be delegated to a shard.
🔒 Do not wrap this skill in
/loop,/schedule, orCronCreate. It is verdict-bearing — it emits a per-reference verdict that gates.bibedits (an evidence instrument, not a cross-model acquittal; see the acceptance-gate note in Step 2). Re-running it on a timer adds no signal (the verdict changes only when the bibliography changes). Schedule the external wait that precedes it: bibliography finalized → run once. See../shared-references/external-cadence.md.
Why this exists (and how it differs from citation-audit)
citation-audit is a single cross-model reviewer that judges three things holistically:
existence, metadata, and context-fit (is the cited paper used for a claim it
actually supports?). This skill does not do context-fit. It does one thing that a
single reviewer cannot do reliably: an adversarial, field-level diff (double-blind when the host can isolate the web agent, a labelled flag-only heuristic otherwise)
of the local .bib entry against an independently reconstructed record.
The failure mode it targets is the hand-entry error that survives a plausibility read:
a dropped or re-ordered author, a wrong year, a swapped venue, a DOI that
404s. A single reviewer that sees the .bib entry tends to anchor on it and rate it
"looks correct." Two agents that never see each other's output cannot anchor — a
discrepancy only survives if both independent records agree, and the verifier reports
exactly where they diverge.
In practice it catches dropped/re-ordered authors, wrong years, and bad DOIs that a
single-pass audit rated "looks clean" — while NEEDS_WEBCHECK stops it from
false-flagging a legitimate non-arXiv (blog / tech-report) citation as fabricated. Use it
in addition to citation-audit, not instead: citation-audit protects your review
score (context-fit); this protects
against the integrity red flag of a wrong or fabricated reference (existence + fields).
citation-audit | citation-crosscheck (this) | |
| Owns | existence + metadata + context-fit | existence + field accuracy, adversarially |
| Method | one cross-model reviewer | two extractors (blind when isolation is attested) + a mechanical field-diff |
| Reads the .tex claims? | yes | no (bibliographic only) |
| Deliverable | KEEP/FIX/REPLACE/REMOVE per entry | side-by-side .xlsx + MATCH/MINOR/MISMATCH |
| Best at catching | wrong-context citation | dropped/re-ordered author, wrong year/venue/DOI |
Constants
- BLIND = conditional — the web agent (B) MUST be prompted with only each paper's title +
first author, plus its opaque
dedup_key(a run-local ordinal that carries no bibliographic content), and MUST NOT read the.bibor any repo file. B attestsblind:trueonly when the host isolates its context so it cannot inherit the.bib(see the mirror-adaptation note above); otherwise B attestsblind:falseand the Step-3.5 gate certifies nothing (flag-only). Do not relax the prompt restriction, and never assertblind:truewithout real isolation — an un-isolatedblind:trueis the one thing that defeats the whole cross-check. - SOURCES = arXiv (
/abs/<id>,/bibtex/<id>), DBLP, CrossRef, ACL Anthology, NeurIPS/PMLR proceedings, OpenReview. Report what the fetched page shows; never fabricate a field. - VERDICTS (assigned by the main agent only, in Step 4 — never by a shard) =
MATCH(bibliographically equivalent, cosmetic-only) /MINOR(formatting or a preprint-vs-published venue, no correction required) /MISMATCH(real discrepancy: wrong/missing author, wrong year, wrong venue, wrong id) /NEEDS_WEBCHECK(B could not confirm existence, but the entry is a@misc/blog/tech-report whoseurlB's index-based search structurally cannot see — the main agent must fetch thaturlin Step 4 before any verdict; never downgrade such an entry straight toMISMATCH). - SCOPE = cited-only by default (entries that render in the
.bbl);--allalso covers uncited.bibentries. - OUTPUT =
<paper-dir>/citation_crosscheck_<date>.xlsx(+ the four intermediate JSONs —cx_A_ourbib,cx_B_web,cx_C_fielddiff, and the gate'scx_gate.json— kept for audit).
Workflow
Step 1 — Resolve the citation set (deterministic, main agent)
# Search the paper's own source tree, not the whole repo: bundled author-kit /
# template .tex carry demo \cite{...} that inflate the set. Set PAPER_DIR to the dir of
# the main .tex (\documentclass + \bibliography); recurse (cites often live in
# sections/*.tex); exclude vendored/example dirs.
PAPER_DIR="<dir of the paper's main .tex>"
cd "$PAPER_DIR"
python3 - <<'PY'
import re, pathlib
# Skip vendored/example/template trees: bundled author kits carry demo \cite{...}.
# NOTE: this is a NAME match — a dir you *want* audited must not match (rename it or
# narrow SKIP), e.g. `examples_of_results/` would be skipped.
SKIP = re.compile(r'(?i)(^|/)(.*template.*|.*upstream.*|.*example.*|.*vendor.*)(/|$)')
# Verbatim-ish blocks AND inline \verb|...| / \lstinline are code samples, not citations.
VERB = re.compile(r'\\begin\{(verbatim|lstlisting|minted|Verbatim)\}.*?\\end\{\1\}', re.S)
# The optional arg may itself contain brackets: \lstinline[language={[LaTeX]TeX}]|...|
VERBINLINE = re.compile(r'\\(?:verb|lstinline)\*?'
r'(?:\[(?:[^\[\]]|\[[^\]]*\])*\])?\s*'
r'([^\sA-Za-z])(?:(?!\1).)*\1', re.S)
# url/href/path args may contain a literal % that is NOT a comment. Neutralize them
# BEFORE comment-stripping, or a real \cite later on that line is silently lost.
URLCMD = re.compile(r'\\(?:url|href|path|nolinkurl)\s*\{[^{}]*\}')
COMMENT = re.compile(r'(?m)(?<!\\)%.*$')
# EXPLICIT ALLOWLIST of cite commands. Do NOT build this from a prefix/suffix pattern:
# a Cartesian `(no|paren|auto|...)?cite(s|p|t|...)?` both invents commands (\citecite,
# \parenciteyear -> fake keys from any brace arg) and misses real ones (\footfullcite,
# \volcite, \notecite). Longest-first so \footfullcite wins over \footcite.
# volcite-family: the FIRST brace arg is a volume/page, NOT a cite key.
VOLCITE = {'volcite', 'Volcite', 'pvolcite', 'Pvolcite', 'fvolcite', 'ftvolcite',
'svolcite', 'tvolcite', 'avolcite',
'volcites', 'Volcites', 'pvolcites', 'Pvolcites', 'fvolcites', 'svolcites',
'tvolcites', 'avolcites', 'ftvolcite', 'ftvolcites'}
NAMES = sorted({
# natbib / LaTeX core (capitalized variants are real natbib commands)
'cite', 'Cite', 'citep', 'citet', 'citealp', 'citealt', 'citenum', 'citeyear', 'citeyearpar',
'citeauthor', 'citefullauthor', 'Citep', 'Citet', 'Citealp', 'Citealt', 'Citeauthor',
# biblatex singular + plural
'parencite', 'Parencite', 'footcite', 'footcitetext', 'footcitetexts', 'textcite',
'Textcite', 'smartcite', 'Smartcite', 'supercite', 'autocite', 'Autocite', 'fullcite',
'footfullcite', 'notecite', 'Notecite', 'pnotecite', 'Pnotecite', 'fnotecite',
'citetitle', 'citeurl', 'citedate', 'autocites', 'parencites', 'textcites',
'smartcites', 'supercites', 'footcites', 'cites', 'Cites', 'nocite', 'nocites',
# aliases + field/list forms + bibentry — real biblatex/natbib families
'citetalias', 'citepalias', 'Citetalias', 'Citepalias', 'citefield', 'citelist',
'bibentry', 'fullciter',
*VOLCITE,
}, key=len, reverse=True)
# (?![A-Za-z]) stops \cite matching the \citeXYZ of an unknown command; the trailing run
# accepts any mix of [opt] and {grp} so \cites{k1}[note]{k2} yields BOTH keys.
CITE = re.compile(r'\\(?:' + '|'.join(map(re.escape, NAMES)) + r')\*?(?![A-Za-z])'
r'((?:\s*(?:\[[^\]]*\]|\{(?:[^{}]|\{[^{}]*\})*\}))+)', re.S)
# Strip [optional] args BEFORE reading brace groups, or a braced note like
# \cite[see {Smith} p.3]{k} contributes 'Smith' as a phantom key.
OPTARG = re.compile(r'\[[^\]]*\]', re.S)
GROUP = re.compile(r'\{((?:[^{}]|\{[^{}]*\})*)\}', re.S)
CMDNAME = re.compile(r'\\([A-Za-z]+)')
# \addbibresource[opts]{x.bib} takes an optional arg; \bibliography{a,b} may list several.
BIBRES = re.compile(r'\\(?:bibliography|addbibresource)\s*(?:\[[^\]]*\])?\{([^}]+)\}')
keys, bibs, nocite_all = set(), [], False
for p in sorted(pathlib.Path('.').rglob('*.tex')):
if SKIP.search(str(p.parent)):
continue
src = p.read_text(errors='replace')
src = VERB.sub(' ', src)
src = VERBINLINE.sub(' ', src)
src = URLCMD.sub(' ', src)
src = COMMENT.sub('', src)
for res in BIBRES.findall(src):
bibs += [r.strip() for r in res.split(',') if r.strip()]
for m in CITE.finditer(src):
raw_cmd = CMDNAME.match(m.group(0)).group(1)
cmd = raw_cmd.lower()
grps = GROUP.findall(OPTARG.sub(' ', m.group(1)))
if raw_cmd in VOLCITE and len(grps) > 1:
# volcite takes {volume}{key}; the PLURAL volcites alternates
# {vol}{key}{vol}{key}... — so keep the odd-indexed args, not just grps[1:],
# or the second volume number is harvested as a phantom key.
grps = grps[1::2]
elif raw_cmd in ('citefield', 'Citefield', 'citelist', 'Citelist',
'bibentry') and len(grps) > 1:
# These commands take a single key(s) arg. If a stray second brace group
# follows (a defensive read of an ambiguous source), do not harvest it as a
# phantom key ('\citelist{k}{publisher}' -> just 'k', not '{k, publisher}').
grps = grps[:1]
for grp in grps:
toks = [''.join(g.split()) for g in grp.split(',')]
# \nocite{*} AND \nocite{*,named}: the wildcard renders EVERY bib entry.
if cmd.startswith('nocite') and '*' in toks:
nocite_all = True # see the NOCITE_ALL note below
for k in toks:
if k and k != '*':
keys.add(k)
print(f"BIB_RESOURCE={bibs[0] if bibs else '(none found - use the fallback below)'}")
print(f"BIB_RESOURCES_ALL={bibs}")
print(f"NOCITE_ALL={nocite_all}")
print(f"CITED_KEY_COUNT={len(keys)}")
import pathlib
# Persist the harvest so the shell can read it back deterministically (the heredoc only
# PRINTS to stdout; it cannot set a parent-shell variable).
pathlib.Path('/tmp/cx_cited_keys.txt').write_text('\n'.join(sorted(keys)))
pathlib.Path('/tmp/cx_cited_count.txt').write_text(str(len(keys)))
print('\n'.join(sorted(keys)))
PY
# Set $CITED_KEY_COUNT as a REAL shell variable (Step 3.5 passes it as --expect): the
# heredoc above only prints it, so read it back from the file the harvest just wrote.
CITED_KEY_COUNT=$(cat /tmp/cx_cited_count.txt)Cross-check against a compiled main.bbl by KEY SET, not count: a count match with a
substituted key still hides a harvest miss (drop one real cite, gain one stray → same count).
Extract the rendered keys and diff the sets:
# \bibitem[label]{key} (natbib/plain) OR \entry{key}{type}{} (biblatex) — key is the first
# brace group either way. Empty diff = harvest and .bbl agree.
grep -oE '\\(bibitem|entry)\*?(\[[^]]*\])?\{[^}]+\}' main.bbl 2>/dev/null \
| sed -E 's/.*\{([^}]+)\}$/\1/' | sort -u > /tmp/cx_bbl_keys.txt
comm -3 <(sort -u /tmp/cx_cited_keys.txt) /tmp/cx_bbl_keys.txt # any output ⇒ investigateThe .bbl cross-check is conditional — an uncompiled submission has none. When absent,
confirm CITED_KEY_COUNT is plausible for the paper's length before proceeding; the
Step-3.5 manifest is the load-bearing coverage guard either way.
If NOCITE_ALL=True the .tex harvest is NOT the citation set. \nocite{*} renders
every entry in the .bib, so under the default cited-only SCOPE the set becomes all
entries that render: take the keys from the compiled .bbl when one exists, else from the
.bib itself (equivalent to --all), and say so in the report. Do not proceed on the
\cite-only harvest — it under-audits by exactly the uncited entries.
BIB_RESOURCES_ALL lists every \bibliography/\addbibresource found, in sorted-path
order; BIB_RESOURCE is just the first. If the list has more than one entry, pick the one
the main .tex declares rather than trusting the ordering, and audit the union only if
the paper really loads several. If BIB_RESOURCE comes back empty (e.g. a \input-flattened
submission), fall back to the largest *.bib under the paper dir and, when a compiled
main.bbl exists, read the cited keys from its \bibitem{...} entries instead of the .tex
(the .bbl is the ground truth for what actually rendered).
Produce the ordered list of cited keys and, for each, its title + first author (the
only thing agent B is allowed to see). Confirm every key exists in the .bib. Assign each
entry a stable opaque ordinal dedup_key (01, 02, … in resolved order): this is the
canonical id that joins the shards downstream. A receives dedup_key + the cite key; B
receives dedup_key + title + first author only (never the cite key or .bib text), so
the join key never leaks the blind side. Write two artifacts for the Step-3.5 gate, in
resolved order: /tmp/cx_dedup_keys.txt (the ordinal set, whitespace-separated) and
/tmp/cx_keymap.tsv (one dedup_key<TAB>cite_key line per entry — the ordinal→cite-key
pin). Step 3.5 builds a coverage manifest (ordinal → cite key → title hash) from
/tmp/cx_keymap.tsv + the .bib and verifies A against it as a mapping, so a citation
dropped or substituted downstream fails the run — a bare count or ordinal-set check would
pass a substitution that keeps the count.
Step 2 — Run A and B blind to each other
This is a read-only extraction fan-out in the sense of
../shared-references/fan-out-pattern.md: A
and B are two shards that only extract records (each returns an entries[] set keyed
by dedup_key — never by list position, never a B-visible cite key, never a quality
ranking). Each shard returns the {shard_id, entries:[{dedup_key, …}]} envelope of
../shared-references/fan-out-pattern.md, and the
executor runs the mechanical dedup_key join below before C. No shard assigns a
MATCH/MINOR/MISMATCH/NEEDS_WEBCHECK label at all — the main agent assigns all
four in Step 4 from C's field evidence, and only its own web-confirm edits the .bib.
Dispatch (tier-portable). On this host there is no Claude Agent tool, so the mainline's
Tier 1 (Workflow-concurrent) and Tier 2 (Agent) forms both degrade here: run A, then B, then
C one after the other, each in the freshest context the host can give. The join and the
field-diff are identical to the mainline. Blindness is not automatic here (see the mirror
adaptation note above): B sets blind:true only if its context provably did not inherit the
.bib, and blind:false otherwise — in which case Step 3.5 downgrades every accept to
ESCALATE and the run flags rather than certifies. B is prompted with only the
title+first-author list and is forbidden from reading any local file; but when B is not run in
an isolated context, that prohibition is not enforced (B could observe the parent's .bib
read, or A's output), which is exactly why an un-isolated B must attest blind:false and the
gate then certifies nothing. Isolation is the guarantee; the prompt restriction alone is not.
Breadth. The shard set is fixed at three by the design, not by a budget: two records are the minimum for a double-blind diff (one per provenance), and the third is the differ that must have produced neither. Breadth is not a swept parameter here, so there is no breadth/cost trade-off to bound — but note the ceiling: two independent records, not an ensemble.
Acceptance gate (provenance, per
../shared-references/acceptance-gate.md). This skill is not a loop and has no self-terminating accept gate; it runs once over a fixed citation set. Type-A (executor may self-judge): thededup_keyjoin balanced, every entry labelled, the.xlsxwritten. Type-B (merit — is the entry bibliographically correct): never delegated to a shard. AMISMATCHor aNEEDS_WEBCHECKresolution requires the main agent's own primary-source fetch — no fetch, no finding. AMATCH/MINORrests on the two independent records agreeing, without a third fetch; that is weaker evidence than a fetch, so read a no-flagMATCHas "two sources concur", not "verified against the publisher of record".Requirement (4) is discharged by the deterministic branch, not by a jury.
fan-out-pattern.mdadmits "a single cross-model jury step — OR a deterministic verifier gate"; this skill takes the second:scripts/cx_verify.py(Step 3.5) is an external process that decidesMATCH/MINORby declared normalization and fails closed toESCALATEotherwise — the same script and the same verdicts on this host as on the mainline.The limit, stated plainly: the accept path is mechanical and a
MISMATCHneeds the main agent's own primary-source fetch, but the executor still reads that page, so this skill does not deliver cross-model acquittal and does not replace one — which is why it is meant to run in addition tocitation-audit(a single cross-model reviewer), not instead of it. Never report aMISMATCH, least of all a fabrication finding, on shard text alone.
- Agent A (repo-only): read the
.bib; extract each cited entry verbatim (brace-matched, no reformatting) →/tmp/cx_A_ourbib.json{"shard_id":"A","entries":[{"dedup_key","key","bibtex"}]}. Explicit instruction: do not search the web. - Agent B (web-only): given ONLY the title+first-author list from Step 1, re-fetch
each paper's record from the web and emit a clean BibTeX per paper →
/tmp/cx_B_web.json{"shard_id":"B","blind","entries":[{"dedup_key","title_queried","found","source_url","bibtex","sources_tried","note"}]}. Explicit instruction: do not read any local file / .bib; setfound:false+ a note if a paper cannot be confirmed; never fabricate a field.blindattestation (set once, for the whole B shard). B setsblind:trueonly if it ran in a context that could not have seen the.bib— a genuinely isolated dispatch that did not inherit the main agent's history. If B cannot guarantee that (e.g. it ran in the main context), it MUST setblind:false. The Step-3.5 gate refuses to certify aMATCH/MINORon any entry that is not blind-attested, so a falseblind:trueis the one thing that would defeat the whole cross-check — never assert it to "pass".- Do not restrict the search to arXiv/DBLP. Many legitimate references are workshop
tech reports, company research blogs, or proceedings pages that never appear on arXiv.
If a title is not on arXiv, B MUST also run a general web search and open the landing
page its own search returns (a project / blog / proceedings URL) before concluding
found:false. B works from the title+first-author query only — it must NOT be handed, and cannot use, the.bibentry'surlfield (that stays with the main agent's Step-4 webcheck). A citation being absent from arXiv is NOT evidence it is fake. - Record
sources_triedper entry (e.g.["arxiv","crossref","semantic-scholar(429)","web"]) so afound:falsefrom a degraded search (a source was down or rate-limited) is distinguishable from a genuine not-found. Note any source that failed/blocked in the entry'snote.
On return, the main agent joins A and B on dedup_key — every ordinal from Step 1 must
appear exactly once on each side. A missing or duplicated dedup_key, or an unbalanced count,
is a fail-closed stop (re-run the offending shard), never a silent drop or a positional
guess. This mechanical join replaces order-coupling and is judgment-free — it decides nothing
about any citation, only that the two extractions line up before C diffs them.
Step 3 — Run C, the field-diff extractor (after A+B return)
C extracts, it does not adjudicate. Computing a field-by-field delta is mechanical extraction; deciding that an entry is
MATCH(i.e. admitting it as correct) is the acceptance verdict, which../shared-references/fan-out-pattern.mdreserves for the executor. So C emits per-field evidence, not a per-entry verdict: for each field it reportssame/differs(with both values verbatim) /absent, plus the routing flags below. The main agent assigns every finalMATCH|MINOR|MISMATCH|NEEDS_WEBCHECKin Step 4 from that evidence. A shard that returns a verdict label is out of contract — the same ruleproof-checkerandresearch-litstate for their shards.
A third pass (a fresh-context run — this host grants no delegation tool) reads both JSONs — it
produced neither — and, on each dedup_key-joined pair, diffs the two records field by
field: title (ignore case/braces),
author set AND order, year, venue (NeurIPS ≡ "Advances in NIPS"; note a genuinely
different event), volume/pages, arXiv id / DOI. Output /tmp/cx_C_fielddiff.json:
{"shard_id":"C","entries":[{"dedup_key","key","blind","found","sources_tried","url","fields":{"<field>":{"status":"same|differs|absent","ours","web"}},"flags":[...]}]}
(found/sources_tried/blind copied verbatim from B, url from A, so Step 4 and the
gate can act without
re-reading the shard files)
(keyed by the joined dedup_key) plus a summary
{entries, fields_by_status:{same,differs,absent}, flags_by_type} — counts of mechanical
field states, not a verdict tally.
Critical routing flag for C (this prevents the most damaging false positive): if B
reports found:false for an entry whose A-side .bib type is @misc (or otherwise
carries a url/howpublished instead of an arXiv id / DOI), and B's sources_tried
shows it only queried indexers that cannot see that URL (arXiv/DBLP/etc.), C MUST attach
the flag url_only_source_unqueried — and MUST NOT report the entry as absent-from-web.
B's index-based method structurally cannot confirm a blog/tech-report citation; only the
main agent's direct url fetch in Step 4 can. C likewise attaches degraded_search when
sources_tried shows a source was down or rate-limited, and web_record_absent whenever B
returned found:false for any reason. These flags force the entry into the main agent's Step-4
queue; none of them is a finding about the citation — they record what B's search did, not
whether the reference is sound.
Step 3.5 — Run the deterministic verifier gate (cx_verify.py, mandatory)
This is the fan-out's requirement-(4) gate: fan-out-pattern.md demands "a single
cross-model jury step — OR a deterministic verifier gate — that is identical across all
three tiers." This skill has no cross-model jury, so the gate is an external process. It is the
same script and the same verdicts on this host as on the mainline — the dispatch of A/B/C
degrades by tier, the gate does not. The gate recomputes both the field diff and the routing
flags from A's and B's own raw records and reads B's blindness attestation from B directly;
C's flags are advisory only (unioned with the recompute) and a C schema violation fails
closed — so C is not in the trust path for the diff or the routing. Resolve $CX_VERIFY
with the Codex hybrid chain
(managed project install, in-repo checkout, ARIS repo fallback, then the copied global Codex
skill install). This is Policy A — skill-local gate:
Resolver note (owner-local, per
../shared-references/integration-contract.md).cx_verify.pyis a single-skill helper under this skill's ownscripts/, resolved via the Codex hybrid chain below. It deliberately ships notools/cx_verify.pyshim — it has no legacytools/entry to forward from (unlike the Phase-3 moves that kept one) — which is the owner-local Layer-0 exception the contract's Policy-A row registers.
CX_VERIFY=""
cd "$(git rev-parse --show-toplevel 2>/dev/null || pwd)" || exit 1
if [ -z "${ARIS_REPO:-}" ] && [ -f .aris/installed-skills-codex.txt ]; then
ARIS_REPO=$(awk -F'\t' '$1=="repo_root"{print $2; exit}' .aris/installed-skills-codex.txt 2>/dev/null) || true
fi
# Global pointer written by smart_update_codex.sh, for hosts with no per-project install.
if [ -z "${ARIS_REPO:-}" ] && [ -f "$HOME/.aris/repo" ]; then
ARIS_REPO=$(cat "$HOME/.aris/repo" 2>/dev/null) || true
fi
[ -f ".agents/skills/citation-crosscheck/scripts/cx_verify.py" ] && CX_VERIFY=".agents/skills/citation-crosscheck/scripts/cx_verify.py"
[ -z "$CX_VERIFY" ] && [ -f "skills/citation-crosscheck/scripts/cx_verify.py" ] && CX_VERIFY="skills/citation-crosscheck/scripts/cx_verify.py"
[ -z "$CX_VERIFY" ] && [ -n "${ARIS_REPO:-}" ] && [ -f "$ARIS_REPO/skills/citation-crosscheck/scripts/cx_verify.py" ] && CX_VERIFY="$ARIS_REPO/skills/citation-crosscheck/scripts/cx_verify.py"
[ -z "$CX_VERIFY" ] && [ -f ~/.codex/skills/citation-crosscheck/scripts/cx_verify.py ] && CX_VERIFY="$HOME/.codex/skills/citation-crosscheck/scripts/cx_verify.py"
[ -z "$CX_VERIFY" ] && {
echo "ERROR: cx_verify.py not resolved at .agents/skills/citation-crosscheck/scripts/, skills/citation-crosscheck/scripts/, \$ARIS_REPO/skills/citation-crosscheck/scripts/, or ~/.codex/skills/citation-crosscheck/scripts/." >&2
echo " The requirement-(4) gate cannot run; do NOT hand-assign MATCH/MINOR in its place. Fix: rerun install_aris_codex.sh, export ARIS_REPO, or copy the citation-crosscheck skill into ~/.codex/skills/." >&2
exit 1
}
python3 "$CX_VERIFY" --selftest || exit 1 # built-in cases; fail = do not trust the gate
# Build the coverage manifest (ordinal → cite key → title hash) from Step 1's ordinal PIN
# (/tmp/cx_keymap.tsv) + the .bib, using the gate's OWN parser. This is independent of A, so
# the gate can verify A against it as a MAPPING. ($BIB = the resolved .bib path from Step 1.)
python3 "$CX_VERIFY" --build-manifest --bib "$BIB" --keymap /tmp/cx_keymap.tsv \
--out-manifest /tmp/cx_manifest.tsv || exit 1
# The gate RECOMPUTES the field diff AND the routing flags from A's and B's own raw records
# (`web_record_absent` from B's `found`, `degraded_search` / `url_only_source_unqueried` from
# B's `sources_tried` + A's fields), and reads B's blindness attestation from B directly. C's
# flags are ADVISORY only, unioned with the recompute, and a C schema violation fails closed —
# so a faulty/omitting/adversarial C can neither hide a discrepancy, forge a MATCH, nor wave an
# entry through by omitting a flag. Coverage is MANDATORY: `--expect-manifest` verifies the
# ordinal→key/title MAPPING against A (catching a drop-and-substitute the ordinal-set check
# cannot); `--expect` / `--expect-keys` add the count and ordinal-set checks. An empty artifact,
# a duplicate/absent/substituted key, a dropped citation, or A/B/C key-sets that disagree all
# exit non-zero rather than reading as "clean".
python3 "$CX_VERIFY" --a /tmp/cx_A_ourbib.json --b /tmp/cx_B_web.json --c /tmp/cx_C_fielddiff.json \
--expect "$CITED_KEY_COUNT" --expect-keys /tmp/cx_dedup_keys.txt \
--expect-manifest /tmp/cx_manifest.tsv \
--json /tmp/cx_gate.json || exit 1What the gate decides, and what it refuses to decide:
MATCH— every compared field equal after declared normalization (LaTeX accent/brace folding, case, punctuation; an order-preserving surname list; a fixed venue-alias table).MINOR— the sole difference is preprint-vs-published venue for the same work.ESCALATE— everything else, fail-closed: any identifying field differing (authors,year,title,doi,arxiv), a secondary field differing, an identifying field absent on one side, a venue pair not in the declared table, any routing flag,found != true, or an entry that is not blind-attested (blind != true). AMATCH/MINORthat lacks the blindness attestation is downgraded toESCALATE: the run can still flag a discrepancy, but it may not certify a match. This is what enforces the double-blind claim mechanically — a host that cannot dispatch B in isolation (so B might have seen the.bib) setsblind:falseand the gate refuses to certify anything, turning the check into a labelled heuristic rather than a false certification.
The gate never emits MISMATCH. A discrepancy becomes a finding only when the main agent
reproduces it against the publisher of record, so the accept path is mechanical and the reject
path is externally evidenced. Author order is preserved by the normalizer on purpose — a
re-ordered author list is one of the defects this skill exists to catch, so it must not
normalize to equal.
Step 4 — ASSIGN every verdict (main agent, mandatory)
No verdict arrives pre-assigned from a shard. Read /tmp/cx_gate.json (the gate's recomputed
verdicts and per-field evidence) — C's cx_C_fielddiff.json is audit-only and must not
override the gate. The invariant of this step: every MATCH/MINOR/MISMATCH rests on
either (a) the gate's mechanical decision or (b) a completed main-agent primary-source fetch.
No shard signal — a differs, an absent, B's found:false, or a routing flag — is itself a
finding. Two exhaustive cases:
- Gate said
MATCHorMINOR→ adopt it (case (a), mechanical). Do not re-litigate a mechanical equality decision from prose, and do not silently upgrade it toMISMATCH; if you believe the gate is wrong, the fix is the gate's normalization table plus a--selftestcase, not a hand-edit of one row. (What a no-flagMATCHmeans: two independently-produced records concur — not "verified against the publisher of record".) Gate said
ESCALATE→ the main agent MUST complete a primary-source fetch before it may assign any ofMATCH/MINOR/MISMATCH/NEEDS_WEBCHECK. This is a procedure, not a first-match menu: do every applicable fetch, then decide — so no earlier branch can preempt a stricter one.- Fetch. Identify and
WebFetchthe primary source(s):- the entry's own
url/howpublishedwhenever the.bibcarries one — always for theurl_only_source_unqueried/web_record_absentcases (this is the.bib'surl, which B never saw); and - the canonical index the fields point at — the arXiv abstract for an arXiv id, the DOI resolver for a DOI, else the venue / proceedings page. For an unconfirmed-existence entry, also run the main agent's own general web search (independent of B). Record every URL and query tried.
- the entry's own
- Decide from what the fetch(es) returned — each terminal rests on a completed fetch:
- Primary source confirms the
.bib(title + author set/order + year + id agree) →MATCH(annotate the source; for aurl-confirmed non-arXiv entry note "blog/tech-report, web-confirmed"). A gatediffers/absentthe source does not reproduce is downgraded here, with the reason recorded. - Same work, only preprint-vs-published differs →
MINOR. - Primary source contradicts an identifying field (author set/order, year,
venue-as-different-event, arXiv id / DOI, or a substantive title/subtitle/word change;
or a moved page/volume range) →
MISMATCH, citing the fetched value. (Field-discrepancyMISMATCH— the source reproduced it; this guards against a B/C hallucination inventing a wrong "fix.") - Existence unconfirmable →
MISMATCH(fabrication) only when BOTH hold: the entry'surlis absent / dead / contradicting, and the main agent's own general search also failed. A deadurlalone, or B'sfound:falsealone, is never enough — this is the one path to a fabrication finding and it rests on the main agent's own null result, not on B's miss. - Fetch inconclusive (source unreachable / genuinely ambiguous) →
NEEDS_WEBCHECK, left for a human. Never default an inconclusive fetch toMATCHorMISMATCH.
- Primary source confirms the
A one-sided
absentfield is not resolved "from the fields both sides share": the gate already escalated it, and the verdict comes from the fetched primary source per the ladder above (an absence becomesMISMATCHonly when the.bibasserts a field the source contradicts; otherwiseMATCH/MINORper the fetch, with the absence noted inDetail).- Fetch. Identify and
Every entry leaves Step 4 with exactly one of the four labels, each MATCH/MINOR/
MISMATCH backed by the gate or a completed main-agent fetch. If an ESCALATE's evidence
fits none of the outcomes above, that is a fetch not yet done, not licence for a shard to
decide. Do not edit the .bib on shard evidence alone: every applied fix traces to a
main-agent fetch.
Step 5 — Emit the spreadsheet + apply confirmed fixes
Build the .xlsx (openpyxl) with one row per citation and columns:
# | BibKey | Link | Our BibTeX | Web BibTeX | B source | Verdict | Author check | Year |
Detail | Fix. Color the Verdict cell: green = MATCH, yellow = MINOR or a
NEEDS_WEBCHECK the main agent resolved to MATCH (annotate how), orange = MISMATCH.
Add a Summary sheet with the counts and the fixed/optional lists. Apply only the main-agent-confirmed MISMATCH
fixes to the .bib; if the paper is compiled, rebuild so the .bbl picks them up and
re-verify the page count is unchanged. Report MINOR items as optional (do not auto-apply
preprint→published upgrades; that is an author choice).
Key rules
- Blindness is the mechanism. If B ever sees the
.bib, the cross-check collapses into a single-source read and loses its power. Enforce it in the prompt. - Never fabricate a field in A, B, or C.
found:false+ a note beats a guess. - A non-arXiv miss is not a fabrication.
@misc/blog/tech-report/proceedings-URL citations are legitimate and invisible to arXiv/DBLP search. B must try the general web from the title+first-author prompt only; C flags an unconfirmed URL-bearing entry (url_only_source_unqueried) and the main agent assignsNEEDS_WEBCHECKand fetches the entry's ownurlin Step 4. AMISMATCH(fabrication) requires both a dead or contradictingurland the main agent's own general web search also failing (per Step 4) — a deadurlalone is never enough. This is the skill's most dangerous false-positive if skipped. - Shards extract; the main agent adjudicates. A, B and C emit records and per-field
deltas. Every
MATCH/MINOR/MISMATCH/NEEDS_WEBCHECKis assigned by the main agent in Step 4, and everyMISMATCHis web-confirmed by the main agent before the.bibis touched. The shards propose; the main agent's own fetch disposes. - Bibliographic only. Context-fit ("is it cited for a claim the paper supports?") is
out of scope — that is
citation-audit's job. Recommend running both before submission. - Report, do not silently upgrade. Preprint-vs-published and cosmetic key/year-label mismatches are MINOR; list them, leave the choice to the author.
Trending
Steady: 0 new projects took it in the last 4 weeks, none the 4 weeks before.thin data
Who's talking about wanshuiyin/Auto-claude-code-research-in-sleep
- Github Signals“What if your AI could brainstorm ideas, plan experiments, review results, and draft papers”YouTube · Aug 4, 2026 · 1k views
- Thành AI“"ARIS" Để AI làm nghiên cứu khoa học suốt đêm, sáng dậy có sẵn paper đã sửa và thí nghiệm đã chạy”YouTube · Aug 8, 2026 · 783 views
- safesynt“ARIS runs ML research workflows while you sleep”YouTube · Jul 17, 2026 · 420 views
- Stack Insider“This free Claude Code setup does your research while you sleep”YouTube · Oct 14, 2026 · 416 views
- AITechZco Pulse“I Let Claude Run ML Research All Night While I Slept”YouTube · Sep 2, 2026 · 287 views
The biggest 5 of 44 mentions linking to the repository that our crawl found. Each line is their own headline.
See all 44
Talked about on GitHub, YouTube and the web (44 times).
On GitHub · 23
- Oct 28, 2026claude-code-marketplaces.github.wanshuiyin.auto-claude-code-research-in-sleep: init at 0.4.28 — sudosubin/agents.nix0 reactions
- Oct 20, 2026Exclude arXiv 2605.03042 (ARIS): autonomous research harness, outside the computer-use scope — reacher-z/Awesome-CUA0 reactions
- Oct 18, 2026基线复用:ARIS原生与Pi移植、强单Agent及数值对照 — LittleDrinks/research-world0 reactions
- Oct 4, 2026agent-skills.github.wanshuiyin.auto-claude-code-research-in-sleep: init at 0.4.28 — sudosubin/agents.nix0 reactions
- Oct 2, 2026docs: document a canonical install directory with symlinks into the agent directory — Boom5426/Nature-Paper-Skills0 reactions · 3 comments
18 more on GitHub
- Jul 17, 2026Caso externo (ARIS) + 3 cicatrices propias: watchdog de progreso y verificador determinista para la mordida — zigiella/cambium-charter0 reactions
- Jun 17, 2026Add ARIS (Auto-Research-In-Sleep) autonomous ML research toolkit — goupup-ai/miccai250 reactions
- May 28, 2026docs/lanes: add ARIS-style coordination contracts (#28) — SlanchaAI/slancha-mesh0 reactions · 2 comments
- May 28, 2026Adopt ARIS-style SKILL.md + 6-state verdict schema for lane coordination — SlanchaAI/slancha-mesh0 reactions
- May 28, 2026Evaluate: ARIS — Cross-Model Adversarial Review für Task-Qualität (⭐10.9K) — greeves89/AI-Employee0 reactions · 2 comments
- May 6, 2026feat: bulk import ARIS skills from local directory or GitHub — zhou-changjian/DeepSeek-TUI0 reactions · 1 comment
- Apr 21, 2026[SUBMISSION] Auto-claude-code-research-in-sleep — ProSkillsMD/proskills0 reactions
- Apr 15, 2026[SUBMISSION] Auto-claude-code-research-in-sleep — ProSkillsMD/proskills0 reactions
- Apr 13, 2026Audit candidate: wanshuiyin/Auto-claude-code-research-in-sleep — xiaolai/nlpm0 reactions · 2 comments
- Apr 10, 2026[SUBMISSION] Auto-claude-code-research-in-sleep — ProSkillsMD/proskills0 reactions
- Apr 9, 2026[Feature]: Sync latest ARIS auto-evolution workflow into Auto Research Hub — OpenLAIR/dr-claw0 reactions
- Apr 6, 2026Audit candidate: wanshuiyin/Auto-claude-code-research-in-sleep — xiaolai/nlpm0 reactions · 2 comments
- Apr 4, 2026[SUBMISSION] Auto-claude-code-research-in-sleep — ProSkillsMD/proskills0 reactions · 12 comments
- Mar 28, 2026[SUBMISSION] Auto-claude-code-research-in-sleep — ProSkillsMD/proskills0 reactions · 23 comments
- Mar 20, 2026[SUBMISSION] Auto-claude-code-research-in-sleep — ProSkillsMD/proskills0 reactions · 2 comments
- Mar 12, 2026Add skill: wanshuiyin/Auto-claude-code-research-in-sleep — VoltAgent/awesome-agent-skills0 reactions · 1 comment
- Mar 12, 2026Add Auto-claude-code-research-in-sleep — e2b-dev/awesome-ai-agents0 reactions
- Mar 12, 2026Add Auto-claude-code-research-in-sleep skill — ComposioHQ/awesome-claude-skills0 reactions
On YouTube · 17
- Aug 4, 2026What if your AI could brainstorm ideas, plan experiments, review results, and draft papers — Github Signals1k views
- Aug 8, 2026"ARIS" Để AI làm nghiên cứu khoa học suốt đêm, sáng dậy có sẵn paper đã sửa và thí nghiệm đã chạy — Thành AI - Rainmaker783 views
- Jul 17, 2026ARIS runs ML research workflows while you sleep — safesynt420 views
- Oct 14, 2026This free Claude Code setup does your research while you sleep — Stack Insider416 views
- Sep 2, 2026I Let Claude Run ML Research All Night While I Slept — AITechZco Pulse287 views
12 more on YouTube
- May 7, 2026Yapay zekanın kusursuz bilimsel yalanları nasıl çözüldü? #ARIS #AI — Fatih Mert Doğancan256 views
- Jun 16, 2026Claude Code, Codex ChatGPT auto-claude-code-research-in-sleep automatiza pesquisa #Claude #Shorts — RepoShots177 views · 1 comment
- May 6, 2026Auto-claude-code-research-in-sleep - GitHub Trending Today — GitHub Trending Digest144 views
- Sep 14, 202616k Star Repo Lets AI Do Your ML Research Overnight #shorts #ai — NaSK111 views
- May 19, 2026Tu IA investiga Machine Learning mientras duermes — lytohlg AI86 views
- Aug 5, 2026寝ている間にAIが研究を回す — Markdownだけの自律リサーチ環境|ARIS 入門【Claude Code / Codex対応・ML研究者向け】 — さつきのOSS研究室66 views · 1 comment
- May 6, 2026ARIS: Open-Source LLM Agents for ML Research — AI Research Roundup39 views
- Oct 23, 2026Claude Code investiga mientras duermes 🌙 ARIS: de 5 a 7,5 en una noche — BarbuPrime IA | Claude Code Vs Codex12 views · 1 comment
- Jun 23, 2026ARIS — AI in the Whitebox - データサイエンスに関わる気になる話題をやさしく解説11 views
- Jun 17, 202612k developers starred this AI tool in 24 hours - wanshuiyin/Auto-claude-code-research-i #Shorts — Observe AI6 views
- Oct 24, 2026This AI Researcher Is Watched by a Rival AI So It Can't Cheat — Harsh Tips3 views
- May 19, 2026Забудь фреймворки! ИИ-исследования в режиме сна 🔥 #Shorts — GitShortAi2 views · 1 comment
On the web · 4
- Oct 29, 2026arxiv | Skills Marketplace — google.com
- Oct 22, 2026wanshuiyin/Auto-claude-code-research-in-sleep - Tokentosea — google.com
- Oct 14, 2026Auto Research In Sleep skill - myagents — myagents.center
- Sep 15, 2026wanshuiyin/Auto-claude-code-research-in-sleep - Aidaegis — google.com
In this repository196 other files it gives its agents.
- ablation-plannerSkill
- alphaxivSkill
- analyze-resultsSkill
- arxivSkill
- auto-paper-improvement-loopSkill
- auto-review-loop-llmSkill
- auto-review-loop-minimaxSkill
- auto-review-loopSkill
- citation-auditSkill
- citation-crosscheckSkill
- claims-draftingSkill
- comm-lit-reviewSkill
- deepxivSkill
- dse-loopSkill
- embodiment-descriptionSkill
- exa-searchSkill
- experiment-auditSkill
- experiment-bridgeSkill
- experiment-planSkill
- experiment-queueSkill
- feishu-notifySkill
- figure-descriptionSkill
- figure-specSkill
- firecrawl-searchSkill
- formula-derivationSkill
- gemini-searchSkill
- grant-proposalSkill
- idea-creatorSkill
- idea-discovery-robotSkill
- idea-discoverySkill
- integrity-forensicsSkill
- interview-cheatsheetSkill
- invention-structuringSkill
- jurisdiction-formatSkill
- keenable-searchSkill
- kill-argumentSkill
- lean-formalizeSkill
- mermaid-diagramSkill
- meta-applySkill
- meta-optimizeSkill
- monitor-experimentSkill
- novelty-checkSkill
- openalexSkill
- overleaf-syncSkill
- paper-claim-auditSkill
- paper-compileSkill
- paper-figureSkill
- paper-illustration-image2Skill
- paper-illustrationSkill
- paper-planSkill
- paper-poster-htmlSkill
- paper-posterSkill
- paper-slidesSkill
- paper-talkSkill
- paper-writeSkill
- paper-writingSkill
- patent-novelty-checkSkill
- patent-pipelineSkill
- patent-reviewSkill
- pixel-artSkill
- The first 60 shown.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

