CLAUDE.md: Openlight

Guidance for Claude Code operating this toolkit. It governs both the shared repository (this repo) and the adjacent institution workspace where the works and local context live. Read it before running any stage.

Openlight: the open, Apache-2.0, Notch8-stewarded toolkit for AI-assisted descriptive metadata generation of digitized cultural-heritage works, with a human-in-the-loop improvement loop. It is the implementation of the architecture in docs/architecture.md.

What this is, in one paragraph

You generate first-pass descriptive metadata for digital works so a professional cataloguer can review and finalize it. You never produce final catalog records. A human reviews everything. The pipeline is seven readable stages and the LLM is only one of them, which is what makes the output auditable. Every value you produce carries confidence, the basis for it, an authority candidate where the value is controlled, and full provenance. The institution owns and tunes the whole thing.

The two-part layout, and precedence

Two directories cooperate. This is the same layering Hyku uses with a local Knapsack over a shared core.

<parent>/
├── openlight/        # this shared repo: generalizable prompts, skills, agents, schemas, docs
└── <institution>-metadata/    # the adjacent workspace, never committed here
    ├── CLAUDE.md              # local: points here, loads the local profile, context, and overrides
    ├── application-profile.json
    ├── works/                 # the actual digital objects (copyrighted or sensitive)
    ├── context/               # collection scope, house style, exemplar good records, donor/provenance notes
    ├── overrides/             # domain-specific prompt/skill/agent customizations (not contributed upstream)
    ├── runs/                  # generation runs, review feedback, pass-rate history
    └── output/                # export-ready records + provenance sidecars

Precedence: local over shared. When an asset exists in both the adjacent overrides/ and this repo, the local one wins. An institution tailors behavior without forking the toolkit. See docs/adjacent-workspace.md for setup.

When you run a job you compose: this repo's assets, plus the local application-profile.json, plus the local context/, plus any local overrides/, with overrides winning.

The hard rules (non-negotiable, from the system prompt)

These hold in every generation, in every institution. They are the trust contract.

1. Never fabricate. If a value cannot be determined from the work or its source metadata, output null and say why in basis. Unknown is correct and preferred over a plausible guess.
2. Confidence on every field. high (directly evident), medium (reasonable inference, check it), low (speculative, must be checked). Default to low when uncertain.
3. Propose authorities, never invent URIs. Give a best-candidate label and set authority_status to needs_verification. A separate deterministic stage (resolve-authorities) confirms it against id.loc.gov, Getty, FAST, VIAF, and others. The LLM proposes; an authoritative lookup disposes; the human confirms.
4. Distinguish what a work IS from what it DEPICTS. A photograph of a building is genre=photograph, type=still image; the building is a depicted subject described with AAT/TGN.
5. Do not name unidentified people. Describe role, clothing, and setting, and flag for human identification.
6. Route sensitive material to a human. Anything potentially harmful, culturally restricted, or requiring reparative description goes in review_flags, not resolved by you. See docs/responsible-description.md.
7. Describe the work in its collection context, not as an atomized item. Use the collection_context (collection, series, parent, source metadata) as evidence, and preserve it on output. See docs/collection-context.md.
8. Record provenance on every record (model, prompt version, date, review status) and emit a PCC-MARC-588-style disclosure_note. See docs/provenance-and-disclosure.md.

Output conforms to schemas/metadata-record.schema.json.

The pipeline (seven stages)

Each stage is a small, readable, version-controlled unit the institution can inspect and modify.

[1] Intake & inventory      list works, capture source metadata and collection hierarchy  (skill: describe-collection)
[2] Pre-processing          derivatives, OCR, AV transcription (local Whisper)             (skill: preprocess-derivatives)
[3] Classification & routing detect content type, pick schema + prompt set                 (skill: classify-and-route)
[4] Generation (LLM)        first-pass record + confidence + provenance                    (prompt: system/metadata-generation; agent: field-generator)
[5] Authority resolution    deterministic lookups, not the LLM                             (skill: resolve-authorities)
[6] Human review            confidence-gated; cataloguer adjudicates                       (skill: review-triage)
[7] Export & ingest         write to the repository in its native format                   (skill: export-for-repository)

Model is a parameter. A frontier model for quality, or a local open-weight model for sensitive collections where nothing may leave the building. The run log records which, and local_only proves it. See docs/models-and-cost.md.

The improvement loop

On a small sample, raise per-record pass rate above the institution's review-policy threshold (for example Oregon State's roughly 90% needs-no-modification bar) before scaling. A frontier model orchestrates and refines prompts; cost-effective models do the volume generation; humans review; pass rate and clustered failures flow back; the frontier model refines and classifies each change as a generalized improvement (contribute upstream) or a domain customization (stays in overrides/). Refine on one sample, confirm the threshold on a fresh held-out sample, then scale. Full detail in docs/the-improvement-loop.md; run it with harness/run-loop.md.

Operating it

The operator-facing prompts in skills/ and the worked example in examples/met-open-access/ are also the Claude Code crash course the engagement transfers. Typical first session: build the application profile, run a first-pass batch on a bounded sample (stop and sanity-check the first 25), triage by confidence, resolve authorities, review, compute pass rate, refine. Do not process a large batch before a sample has cleared review.

Contributing changes back

When a change would help an institution describing entirely different works, without imposing this collection's assumptions, it is generalizable and belongs upstream. When it encodes this collection's scope, house style, local vocabulary, or donor context, it stays in the adjacent overrides/. See CONTRIBUTING.md for the test and the PR process. Contributing is voluntary and makes the commons better; it never creates a dependency, and the institution owns its tuned instance regardless.