research_LLM_wiki / literature-wiki

# Literature Wiki — Codex CLI entry (Schema v2)

This is the **Codex CLI** entry for the literature-wiki skill. The Claude Code entry is `SKILL.md` next to this file. The two contain the same operational protocol; this file is self-contained because **Codex does not support markdown imports** ([openai/codex#17401](https://github.com/openai/codex/issues/17401)). If you edit one, mirror to the other.

**Schema v2 (current)** introduces controlled vocabulary on source frontmatter and an `update-index` operation that augments concept pages with auto-generated "Papers using this" appendices. **v2.1** adds the source-page gist: every `sources/*.md` opens with a `## For future agents` preamble; pre-v2.1 pages get backfilled on next touch or flagged by `lint`.

## When to act

Activate when the user says any of:

- "加入文献wiki" / "把这篇加进去" / "我读完这篇了" / "归档这篇" / "更新文献库"
- "我之前看过谁的 X 工作" / "我的 wiki 里有没有 X"
- "ingest this paper" / "update the wiki" / "what does my wiki say about X"
- "add to my literature wiki" / "lint the wiki"

**Do NOT activate** for fresh literature discovery, reference recommendations, or DOI/BibTeX lookup unless the user explicitly says "add to wiki", "query my wiki", or "lint the wiki". Delegate to a literature-search tool for discovery; this skill files what the user has read, not what they want to find.

## Why this skill exists

Most LLM-document workflows are RAG: every question re-discovers knowledge from raw PDFs. Nothing accumulates. For a researcher reading dozens of papers a year on related topics, this is wasted compounding.

This skill maintains a persistent, structured wiki of markdown files. New paper → skill reads it, extracts what matters, integrates into existing entity / method / system / observable pages — updating cross-references, flagging contradictions, strengthening synthesis. Wiki is the compiled artifact; raw PDFs sit beside it but are not the lookup layer.

User curates and asks. Skill writes and maintains.

Field-independent. Default categories suit physical-science research; users in other fields rename in `<wiki>/AGENTS.md` (or `CLAUDE.md`).

## Wiki location

Resolution order:

1. `LITERATURE_WIKI_PATH` env var.
2. `~/.literature-wiki-path` (single line, absolute).
3. Default: `~/research-wiki/`.

Ask before creating if absent.

## Wiki layout

```
<wiki>/
├── CLAUDE.md / AGENTS.md      # ⭐ vocabulary + style overrides
├── index.md                   # navigation hub
├── log.md
├── raw/                       # source PDFs/markdown, IMMUTABLE
│   └── assets/
├── sources/                   # one page per paper
├── entities/                  # author / group / experiment / code-package pages (manual + AUTO appendix)
├── methods/                   # method-level pages (manual + AUTO appendix)
├── systems/                   # system-level pages (manual + AUTO appendix)
├── observables/               # quantity-level pages (manual + AUTO appendix)
├── debates/                   # who-disagrees-with-whom
└── synthesis/                 # cross-cutting themes
```

In v2, every concept page (`entities/`, `methods/`, `systems/`, `observables/`) has the structure:

```markdown
# <Canonical name from vocab>

(manual prose: definition, key references, ongoing debates)

<!-- AUTO:BEGIN -->
## Papers using this <axis> (N)
- [YYYY] Author et al. — one-line claim. → [[../sources/<paper-id>]]

## Cross-tags
methods=[…], systems=[…], observables=[…]
<!-- AUTO:END -->
```

Manual section is your encyclopedic content. AUTO block is regenerated by `update-index`.

## Controlled vocabulary (v2 contract)

All tag values come from a controlled vocabulary in `<wiki>/CLAUDE.md` (or `<wiki>/AGENTS.md`). Four axes:

| Axis | What it tracks |
|---|---|
| `entities` | authors, groups, experiments, code packages from the literature |
| `methods` | analytical/numerical/experimental methods used by source papers |
| `systems` | specific reactions / nuclei / datasets the source studies |
| `observables` | physical or computational quantities computed/measured |

Narrower than research-profile's six axes (no `topics`, `codes`, `collaborators` — those are user-specific). When the user runs both skills, **methods/systems/observables slugs should match across both wikis**.

### Vocabulary file format

```yaml
vocabulary:
  methods:
    cdcc:
      canonical: "Continuum-Discretized Coupled Channels"
      aliases: [CDCC, "continuum-discretized"]
    iav-cdcc:
      canonical: "Ichimura-Austern-Vincent CDCC"
      aliases: [IAV-CDCC]
      parent: cdcc
  entities:
    moro-group:
      canonical: "A.M. Moro group, Universidad de Sevilla"
      aliases: ["Moro group", "Sevilla group"]
```

`canonical` required. `aliases` case-insensitive multi-language. `parent` propagates to indices.

### Vocabulary discipline

When ingesting:
1. Look up tags against vocabulary (case-insensitive, alias-aware).
2. Known: use canonical slug.
3. Unknown: STOP. Propose vocab diff with definition + aliases. Apply only after explicit user approval.

Reserved `uncategorized` slug per axis; lint warns at >30%.

## Source page frontmatter

```yaml
---
type: source
created: YYYY-MM-DD
last_updated: YYYY-MM-DD
title: "..."
authors: [list, lead first]
year: YYYY
venue: "..."
doi: 10.xxx/xxx
arxiv: ...

methods: [iav-cdcc, cdcc]
systems: ["d+93Nb"]
observables: [breakup-cs]
entities: [moro-group]

contradicts: [[sources/<other>]]
extends: [[sources/<other>]]
research_profile_links: [[../research-wiki-personal/papers/published/<slug>]]
---
```

Concept page frontmatter is simpler: `type`, `slug`, `created`, `last_updated`, `last_indexed`.

## Source page gist (v2.1)

Immediately after the frontmatter, every `sources/*.md` opens with:

```markdown
## For future agents

<2-3 sentences: what the paper claims (with the one key number), which wiki tags /
research lines it touches, and when this page would matter for a query.>
```

Self-contained (no "see below", name slugs explicitly), concrete (carry the key number), decidable in 10 seconds. This is the triage layer for `query`. (Pattern borrowed from obsidian-second-brain's AI-first vault rule.)

## Operations

### 1. Ingest

Trigger: paper dropped into `<wiki>/raw/`, a PDF or folder of PDFs anywhere on disk, or DOI/arXiv ID, with "归档" / "ingest" / "把这篇加进去".

**Set consolidation depth first.** Payoff concentrates on papers central to the user's active research lines, so scale effort with centrality, not uniformly. Judge centrality (cross-check `index.md`, research-profile if available), then pick the pass:
- *Peripheral* (off the user's lines, likely cited once): light pass = steps 1, 2, 4 with a lean source page, 6, 8. Skip deep cross-linking and concept-page rewrites.
- *Core* (on an active line, likely built on or cited): full pass = all steps, real work at step 5 (rewrite affected concept-page prose, not just append) and step 3 contradiction hunting.

1. **Read source.** If only an identifier given: for an arXiv **nucl-th** paper, FIRST check the local corpus by running `/Users/jinlei/anaconda3/bin/python ~/literature-corpus/query.py show <arxiv_id>` — on a hit the character-exact `.tex` source is already on disk (the printed `tex_dir`, on the KINGSTON drive), use it, no re-fetch. On a miss, or for non-nucl-th / non-arXiv papers, fetch via the user's literature-search tool. Otherwise via Codex web access. (The corpus holds full text but NOT verified BibTeX; citations still go through literature-search.)
   - **PDF-only (no .tex / arXiv source), equations/tables matter:** the raw PDF text layer garbles inline equations. First convert with the pdf-extract wrapper, `~/.claude/skills/pdf-extract/scripts/pdf2md.sh "<pdf path>"` (bare marker on heliumx, prints the markdown path), read that markdown as the scaffold, then verify every load-bearing number/equation against the original PDF. The PDF is still what gets archived into `raw/` at step 4; the markdown is a working scaffold. Wrapper unavailable → read the PDF directly and flag values as unverified. When a .tex / arXiv source exists, skip this and read the source (character-exact).
2. **Discuss.** Surface 3–5 takeaways, ask which to emphasize. Non-skippable on first session ingest.
3. **Vocabulary check.** For each method/system/observable/entity invoked: vocab lookup. Unknown → propose vocab diff first, await approval.
4. **Archive PDF into `raw/`, then create `sources/<paper-id>.md`.** FIRST copy the source PDF into `<wiki>/raw/` if not already there (stable filename). Mandatory for every pass incl. light-bulk: if the paper came from a path outside the wiki, was fetched, or bulk-ingested from an external folder, physically copy the file in, never just reference it; never leave `source_pdf:` pointing outside `raw/`. THEN write the page with full v2 frontmatter, the `## For future agents` gist as the first body section (v2.1, mandatory on every pass incl. light), bibliographic header, abstract in user's words, key claims with section refs, key numbers, figures (downloaded into `raw/assets/`), wikilinks to concept pages; set `source_pdf:` to the archived filename now in `raw/`.
5. **Update or create concept pages** for each tag value. Manual prose updated; AUTO block regenerated by step 7.
6. **Update `index.md`** with one-line summary.
7. **Trigger `update-index`** — regenerate AUTO blocks on touched concept pages.
8. **Append to `log.md`**: `## [YYYY-MM-DD] ingest | <paper-id> | <takeaway>`.
9. **Confirm** changed pages and contradictions. A single paper typically touches 5–15 pages.

**Hard invariant (PDF archival).** Every `sources/*.md` declaring `source_pdf:` MUST have that file in `<wiki>/raw/`. The wiki is the self-contained archive; a page naming a PDF that lives only in an external folder (EndNote/Downloads/Zotero) is a broken ingest. Bites hardest on **bulk ingests via parallel workers**: each worker `cp`s its PDF into `raw/` as part of writing the page, and the orchestrator verifies `raw/` count rose by the number of new pages before reporting success. After any bulk run, audit: every `source_pdf:` resolves to a real file in `raw/` (0 misses), else copy the stragglers.

### 2. Query

Trigger: user wants synthesis across wiki rather than fresh search.

1. Read `index.md` to find candidate concept pages.
2. Triage source-page candidates by their `## For future agents` gist (first body section, cheap); drill into the 3–10 pages whose gist matches, with file:line citations.
3. Synthesize, flag supported / contradicted / gaps.
4. **Offer to file the answer** as a new `synthesis/` or `debates/` page if non-trivial. Compounding.

### 3. Lint

Trigger: explicit, or after every ~10 ingests.

Check: contradictions between sources, stale claims, orphan sources (no axis tags), missing concept pages, frontmatter drift, missing `## For future agents` gist (pre-v2.1 pages; backfill from page content), AUTO-block hand-edits, cross-link validity.

Output: markdown report. Apply only after user reviews.

### 4. update-index (NEW in v2)

Auto after every `ingest`; after `lint` only once the user approves the lint fixes (lint stays read-only). Idempotent.

1. Walk `sources/*.md`, read frontmatter only.
2. Build axis → tag → sources map. Apply `parent:` propagation.
3. For each referenced concept page (`<axis>/<slug>.md`): ensure exists, locate AUTO markers, regenerate AUTO block:
   ```markdown
   <!-- AUTO:BEGIN -->
   ## Papers using this <axis> (N)
   - [YYYY] First-author et al., Venue — one-sentence claim. → [[../sources/<paper-id>]]

   ## Cross-tags
   methods: […], systems: […], observables: […]

   ## Last indexed: YYYY-MM-DD
   <!-- AUTO:END -->
   ```
4. Concept pages with no remaining sources: mark `## Status: orphaned` in manual section header (do not delete; prose may still have value).
5. Report changes.

### 5. migrate (NEW in v2)

For pre-v2 wikis:

1. Walk `sources/*.md`, identify missing v2 fields (tag axes, cross-references). Propose from wikilinks; user confirms.
2. Walk concept pages, insert `<!-- AUTO:BEGIN/END -->` markers if absent (at bottom; existing prose preserved above).
3. Run `update-index` once.
4. Append migration log.

Interactive and slow on large wikis — by design.

## Working principles

- **The user curates, the skill writes.**
- **Quote with provenance.** Every non-trivial claim links back.
- **Lean over comprehensive.**
- **Use `[[wikilinks]]`.**
- **Read `<wiki>/AGENTS.md` first** for vocabulary and style.
- **Controlled vocabulary is a contract.**
- **Frontmatter drives indices, body holds narrative.**

## Cross-skill hand-off

### To/from research-profile (user's own portfolio)

- Shared vocabulary slugs for `methods`, `systems`, `observables`. Recommended: single `~/research-vocabulary.yml`, both wikis' CLAUDE.md/AGENTS.md `include:` it.
- `research_profile_links:` on source frontmatter when user has own paper responding to / extending / contradicting the source.
- Reverse: research-profile pages add `literature: [[../research-wiki/sources/<paper>]]`.

### To/from literature-search

literature-search skill is the source of *new* papers (DB fetcher with anti-hallucination). Output → input here via `<wiki>/raw/`.

### To/from literature-corpus

literature-corpus is the full local arXiv nucl-th corpus (~62k papers, full text, lexical/BM25), this wiki's **local feed-in**. Run `/Users/jinlei/anaconda3/bin/python ~/literature-corpus/query.py {show <id>|text "<q>"|abs "<q>"}`. On nucl-th ingest, `show <id>` locates the on-disk `.tex` source (step 1, no re-fetch); `text`/`abs` discover field-wide related work beyond the wiki. Full text + metadata but NOT verified BibTeX (citations → literature-search). A corpus miss is never proof of novelty (nucl-th-only, lexical).

### To/from paper-writing skills

Paper-writing skills query this wiki for related-work sections. Query returns `[[wikilinks]]` that paper-writing resolves to verified BibTeX.

## Codex install notes

- **Per-project install (recommended):** `cat /path/to/skills/literature-wiki/AGENTS.md >> <project>/AGENTS.md`. Loads in that project.
- **Global install:** append to `~/.codex/AGENTS.md` (NOT if you have research-profile auto-load taking that file via symlink).
- The default `project_doc_max_bytes` is 32 KiB combined. Raise in `~/.codex/config.toml` if needed:
  ```toml
  project_doc_max_bytes = 65536
  ```
- Codex has no skill-trigger mechanism. The "When to act" rules above shape Codex behavior in absence of slash-commands.

## Templates shipped

- `templates/CLAUDE.md` — vocabulary starter with four axes (entities/methods/systems/observables) and example entries. Copy to `<wiki>/CLAUDE.md` on first use.