review-triage

Stage 6 of the pipeline. You present generated records to a professional cataloguer for review, triage the cataloguer's attention by confidence so the flagged minority gets scrutiny and the bulk gets a fast confirmation, and capture a verdict on every required value. The cataloguer adjudicates; you organize the work and write down the result. You never finalize a record on your own.

This skill triages effort within one run. Improving the prompts across rounds so the pass rate clears the threshold is a separate concern handled by compute-pass-rate and refine-prompts. See ../../docs/the-improvement-loop.md for why those are different tools.

Where this sits in the pipeline

[5] Authority resolution  →  [6] Human review (this skill)  →  [7] Export
    resolve-authorities          review-triage                    export-for-repository

You run after resolve-authorities has done its deterministic lookups, so the needs_verification values reaching you are the ones the authoritative lookup could not confirm and left for human judgment, rather than values that were never looked up. You run before compute-pass-rate, which reads the review feedback you produce and computes the per-record gate, the per-field rates, and the calibration check. You do not compute those yourself.

Inputs

• Generated records conforming to ../../schemas/metadata-record.schema.json, already through Stage 5. Each value is a fieldValue carrying value_id, value, confidence, basis, authority_status, and the rest. Records may carry review_flags, a record-level array of free-text escalation notes (sensitive content, reparative description, unidentified people).
• The application profile for the run, which says which fields are required and so which count toward the per-record gate.

Outputs

• Review feedback conforming to ../../schemas/review-feedback.schema.json: one reviewed_records entry per record, with a fields entry per reviewed value carrying field, value_id, status (accepted, edited, rejected), stated_confidence, original, final, and note, plus a record_passed boolean. This is the input to compute-pass-rate.
• The same verdicts written back onto each record's per-value review block and the record-level record_review block in the metadata record, so the record carries its own review state.

Every verdict references a value_id. Never an array position. A verdict on the second subject attaches to subjects.02, the stable id assigned at generation, so it stays attributable when a later prompt change reorders the array. Positional keys are never written anywhere.

The triage procedure

1. Surface what needs attention first

Build a focused view that pulls the flagged minority to the top and hides the rest, in this order:

1. Records carrying review_flags. These are the escalations the generation stage routed to a human on purpose: sensitive or culturally restricted material, reparative-description cases, and unidentified people the model declined to name. Surface the whole record with its flags. Note that review_flags is a record-level array of strings, so it is not attached to any single value.
2. Values with authority_status: needs_verification. The deterministic authority stage could not confirm these against id.loc.gov, FAST, Getty, VIAF, or the profile's vocabularies, so a human decides the authorized term and URI.
3. Values with confidence: low, then confidence: medium.

Group by field so the cataloguer can see where the work concentrates: whether the dates, the subject headings, or the depicted entities carry most of the uncertainty.

2. Batch-approve the high-confidence remainder

The values the cataloguer did not need to scrutinize still get a recorded verdict. Batch-approve writes a recorded verdict for every value it clears rather than skipping them. For each high-confidence required value the cataloguer confirms in bulk, emit a review-feedback entry with status: accepted and the value's stated_confidence (high). Two downstream computations depend on this. The per-record gate cannot be decided without a status for every required field, and the calibration check needs the stated confidence of accepted values to test whether high-confidence values are in fact accepted at a high rate. A batch-approve that recorded nothing would leave the gate uncomputable and hide miscalibration.

Batch-approve is a skim-and-confirm efficiency for the cataloguer. The human still owns every verdict. Stated confidence is a triage signal that the loop tracks rather than trusts, so a high-confidence value is presented for fast confirmation and is never auto-accepted on the model's say-so.

3. Adjudicate each surfaced value

For every required value, record one verdict:

• accepted: the value stands as generated. Set status: accepted.
• edited: the cataloguer changed the value. Set status: edited, put the generated value in original and the corrected value in final. The correction is what later lets a cluster of edits point back to the prompt that produced them.
• rejected: the value is wrong and removed. Set status: rejected with a note.

For a needs_verification value the cataloguer resolves, accepting the proposed authorized label and its confirmed URI is an accepted verdict; choosing a different authorized term is an edit.

4. Handle missing values in required arrays

If the cataloguer finds a required array field (subjects, genre_form, depicted_entities) is missing a value the record should carry, capture the addition as a new fields entry with a freshly assigned value_id (for example subjects.07), the added value in final, and a note explaining it was a cataloguer addition. The review-feedback schema has no missing status, so do not invent one. A cataloguer-added value means the field failed the gate, which is recorded by the gate logic in the next step.

The per-record gate

Compute record_passed for each record from the verdicts, by the definition in ../../docs/the-improvement-loop.md:

• A required scalar field passes when its value is accepted with no edit.
• A required array field passes only when every generated value in it is accepted and the cataloguer added no missing value. One edited, rejected, or missing value fails the field.
• A record passes (record_passed: true) only when every required field passes. Any failed required field fails the record.

Non-required fields are reviewed and their verdicts recorded, and they do not affect record_passed.

Set the matching record_review.status and record_review.passed on the record, and human_reviewed to true in provenance. On approval, the disclosure note is completed with the reviewer and date.

The threshold the institution measures these passes against (for example Oregon State's roughly 90%) is the institution's own review-policy gate and the improvement loop's optimization target. It is not a Notch8 contractual accuracy guarantee.

Usage example

This is sample user prompt #3, confidence-gated review triage. It exercises the surfacing half of the procedure: filter the structured output so the cataloguer sees the uncertain values grouped by field and the high-confidence values are hidden from view.

Load all records in ./output/telegrams/. Show me only the fields marked confidence
"low" or with authority_status "needs_verification", grouped by field type, so I can
see whether the date fields or the subject headings need the most attention. Don't
show me anything marked high confidence.

Expected behavior:

• Read every record in ./output/telegrams/ against ../../schemas/metadata-record.schema.json.
• Build the filtered view: include each value with confidence: low or authority_status: needs_verification; exclude confidence: high. Surface any record carrying review_flags at the top regardless of its field confidences, since those are deliberate human escalations.
• Group the surfaced values by field (date, subjects, depicted_entities, and so on) and report the count per field so the cataloguer can see where attention is owed, identified by value_id.
• As the cataloguer works the surfaced list, record an accepted, edited, or rejected verdict per value into review feedback. When the cataloguer is ready to clear the high-confidence remainder, run the batch-approve from step 2, which records status: accepted for each of those required values so every required field has a verdict and the per-record gate can be computed.

The prompt shows only the filtering. Triage is the filter plus the verdicts: hiding high-confidence fields focuses the human, and batch-approving them with recorded verdicts is what makes the run computable downstream.

Scope boundary

This skill captures verdicts and writes review feedback. It does not compute pass rates or calibration (compute-pass-rate), propose prompt changes (refine-prompts), resolve authorities (resolve-authorities, Stage 5, which has already run), or export records (export-for-repository, Stage 7). It surfaces sensitive and reparative cases for the cataloguer and never adjudicates them itself. See ../../docs/responsible-description.md.