Preprocess Derivatives

Stage in the pipeline

This is Stage 2, Pre-processing, in the seven-stage pipeline described in ../../docs/architecture.md. It runs after describe-collection (Stage 1, intake and inventory) has produced the work inventory with source metadata and collection hierarchy, and before classify-and-route (Stage 3, content-type detection and prompt selection) and the generation stage (Stage 4).

Stage 2 turns a raw work into something the generator can read. It produces the derivatives the rest of the pipeline needs, extracts the text and transcript evidence a vision-only pass would miss, and records the technical facts (format, duration, dimensions) that downstream stages and the human reviewer rely on. It does not generate descriptive metadata and it does not emit a metadata record. Its job is to assemble evidence and hand it forward.

What this skill does

Given the intake inventory from describe-collection (one entry per work, with source metadata and collection context attached), for each work:

1. Detect the modality. Use the file format and the inventory's source metadata to decide what the work is: a still image, a text-bearing image (a scanned document, letter, or telegram), audio, video, or a compound object. The modality decides which of the steps below apply. This is a coarse routing decision; the finer content-type classification (photograph versus artwork, audio versus video) belongs to classify-and-route in Stage 3.
2. Generate derivatives. Produce the access and working derivatives the pipeline needs, for example a downsized image for the generator to read, a normalized page image for OCR, or extracted page images from a multi-page object. Keep derivatives alongside the work in the adjacent workspace; the original is left untouched.
3. Run OCR on text-bearing images. Where the modality is a text-bearing image, run OCR and capture the extracted text as evidence. The generator reads this text rather than inferring content from pixels alone, which is what lets it draft a description and subjects for a document it can actually read. Capture the OCR engine and a rough quality signal so the reviewer knows how much to trust the text; noisy OCR is a known failure cluster the improvement loop tracks.
4. Transcribe audiovisual material locally. Where the modality is audio or video, run transcription through the transcribe-video skill, which uses a local Whisper model. Audio and video never leave the institution's environment. This is the data-governance guarantee for rights-sensitive and donor-restricted AV: the work stays in the building. The skill takes an audio or video file and produces a timestamped transcript (SRT, VTT, or TXT); it is English-default and does not label speakers. The transcript becomes the evidence the generator draws description and subjects from in Stage 4. Treat any named people in the transcript as candidates only; per the hard rules, the generator proposes them and routes them to authority verification, and unidentified people are never named.
5. Read technical metadata. For every work, read the technical facts the format exposes: file format and MIME type, image dimensions, and for AV the duration, and capture them as evidence. These facts inform the Stage 3 routing decision, populate technical descriptors at generation (for example PBCore technical fields for AV), and give the reviewer the format context a description alone omits.
6. Read IIIF manifests where present. Where a work arrives with a IIIF manifest (the IIIF Image and Presentation APIs deliver and present digitized images, manuscripts, and maps), read it for the structural and technical metadata it carries: canvas count and order, image dimensions, and any descriptive or rights statements already recorded. Capture this as additional source-metadata evidence; it is especially useful for compound objects, where the manifest is the most reliable statement of how the pages relate. [verify] the exact manifest fields read against the institution's IIIF profile, since version and local practice vary.

Inputs and outputs, by schema

Stage 2 sits between schema-defined artifacts but does not itself emit a metadata record. The honest mapping:

Inputs

• The intake inventory produced by describe-collection (Stage 1): one entry per work, with the collection hierarchy and any pre-existing source metadata attached. This is the evidence base Stage 2 adds to.
• The institution's application profile, ../../schemas/application-profile.schema.json:
  • target_schema indicates when a work needs AV handling (a PBCore target signals audiovisual material, which triggers local transcription and AV technical metadata).
  • model.local_only_collections is the governance trigger. A work in one of these collections must stay in-house; for AV this is already satisfied because transcribe-video runs locally, and the run must record that nothing left the institution.

Outputs

• Derivatives, OCR text, transcripts, and technical metadata, written into the adjacent workspace alongside each work. These are evidence, not descriptive values.
• This evidence is carried into the metadata record at generation through ../../schemas/metadata-record.schema.json: collection_context.source_metadata is free-form by key (additionalProperties: true) and is the defined home for the OCR text, the transcript reference, and the technical facts this stage extracts. The generator reads source_metadata as evidence and the cataloguer checks the values against it. Stage 2 does not write to the fields object; that is the generator's output in Stage 4. The record's fields set is fixed (additionalProperties: false), so AV technical descriptors that have no defined field land in source_metadata as evidence; carrying them as named PBCore technical fields requires a profile or schema extension. [verify] against the institution's target schema before relying on a specific PBCore field name.
• The run manifest, ../../schemas/run-manifest.schema.json, records the run. When AV or any local-only work was processed without anything sent to an external API, set models.local_only to true so the run carries proof of the data-governance guarantee.

Usage example

The audiovisual collection from operator prompt #6 in the initiative's sample user prompts:

This collection is digitized oral-history audio (./collections/oral-histories/).
Run the transcription stage locally first (Whisper, nothing leaves this machine),
then generate metadata using the PBCore target schema instead of Dublin Core, pulling
description and subjects from the transcript. Flag any named people for LCNAF verification.

This skill owns the first half of that request: run the transcription locally through transcribe-video, confirm nothing left the machine, and read the AV technical metadata (format, duration). For each oral-history file it produces a transcript and a technical-metadata record, captured as evidence in collection_context.source_metadata, and it sets the run's models.local_only flag.

The rest of the prompt is downstream. Selecting the PBCore target schema and routing the works as audio is classify-and-route (Stage 3). Drafting description and subjects from the transcript, proposing named people as creator or depicted_entities candidates with authority_status needs_verification for LCNAF resolution, is the generation stage (Stage 4). Run this skill first, then hand the transcripts and technical metadata forward.

What this skill does not do

• It does not classify content type or select the target schema and prompt set. That is classify-and-route (Stage 3).
• It does not draft any descriptive value, propose authority candidates, or assign confidence. That is the generation stage (Stage 4).
• It does not name people from a transcript or an image. Per the hard rules, the generator proposes named entities as authority candidates and routes unidentified people to human review.
• It does not send audio or video to an external API. AV transcription is local by design.

Handoff

When derivatives, OCR text, transcripts, and technical metadata are captured for the works in the batch and attached as evidence, the works are ready for classify-and-route (Stage 3), which detects content type and selects the schema and prompt set, and then the generation stage (Stage 4), which drafts the first-pass record from the evidence this stage assembled.