mif-corpus

Shared substrate for semantic discovery over MIF documents. Where mif-validate proves what a document IS (deterministically), this skill finds what a document is ABOUT — a local-embedding vector index queried by meaning, not string match. It is backed entirely by the optional mif-rs tools and is an enhancement layer: nothing in this suite's conformance path depends on it.

Tool resolution order (never fake results)

1. MCP tools — when an mif-mcp server is connected in the session, use its tools: ingest_mif_document, search_documents, find_similar_documents, corpus_stats (and validate_mif_document, resolve_ontology_reference for the non-corpus operations).
2. CLI fallback — when only the mif-cli binary is on PATH, use the matching subcommands: ingest, search, find-similar, corpus-stats.
3. Neither available — say so plainly, point at the install route (attested release binaries from modeled-information-format/mif-rs, or cargo install mif-cli mif-mcp), and stop. Do not simulate scores, invent matches, or substitute grep output labeled as semantic search.

Store convention

• The vector store lives at .mif/vectors.db, created on first ingest. The tools' default is relative to the working directory — run them from the repo root so the store lands at <repo>/.mif/vectors.db (or pass --db-path / the db_path parameter to pin it explicitly; a run from a subdirectory otherwise creates a second store silently). It is gitignored — a derived local index, never committed, never an authority.
• Document ids are the MIF @id (urn:mif:<id> from the frontmatter id).
• Re-ingesting a changed document upserts its embedding; ingest is fail-closed — a document that fails schema validation or the lossless round-trip stores nothing.

Operations

Search and similar results are (score, id) pairs, most similar first. Scores are cosine similarity; on this suite's own corpus, useful matches typically land in the ~0.55-0.78 range. Treat results as candidates for a human or a genre skill to act on — never as a validation verdict.

Multi-root queries (spanning a shared central store)

search/find-similar/corpus-stats (CLI) and their MCP counterparts search_documents/find_similar_documents/corpus_stats accept an additional, repeatable root: --extra-db-path <path> on the CLI, an extra_db_paths array parameter on the MCP tools. This lets a query span the project-local .mif/vectors.db and any number of other vector stores in one call — most commonly a shared, user-level central corpus that multiple projects ingest into (e.g. the claude-artifact-authoring plugin's persist-artifact skill best-effort-ingests graded artifacts into one via mif-corpus ingest, at an XDG-conformant path — $HOME/.local/share/claude-artifact-authoring/corpus/vectors.db, i.e. ${XDG_DATA_HOME:-$HOME/.local/share} with the default expanded).

• Results are merge-ranked by cosine similarity across every root for search/find-similar — not grouped or root-ordered — with each match additionally tagged with the root it came from. Single-root behavior (no --extra-db-path/extra_db_paths given) is unchanged.
• corpus-stats/corpus_stats is not a ranked search — with extra roots it returns the primary root's count/dim plus a total_count and a per-root breakdown (extra_roots); nothing is merge-ranked, since there is no query vector in a stats call.
• Fails closed per root: a root that cannot be opened or queried aborts the whole call rather than silently dropping it from the results. A root that has a valid parent directory but no database file yet (never ingested) is treated as empty, not an error.
• The same document id can legitimately exist in more than one root (e.g. a document ingested both locally and into the shared central store) — this is expected, not a collision to dedupe.

Known exclusion: documents with a description: key (the ADRs)

Documents carrying a top-level description: frontmatter key currently fail the Rust round-trip check inside ingest (the key is dropped by the canonical re-serialization; tracked upstream and in this repo's engine-convergence epic). In this suite's corpus those are exactly the ADR documents under docs/adr/, which carry MIF type: adr (conceptType: semantic), so the skip set is identifiable by the description: key, the docs/adr/ path, or the type: adr frontmatter value alike. Skip them in bulk ingests and say that they were skipped and why. This sits alongside the suite's existing rule that the adr genre is validated by the structured-madr Action, not mif-validate.

First-run cost

The first ingest or search downloads the sentence-embedding model into the local hf-hub cache — it needs network access once and takes noticeably longer than subsequent runs. Vectors are 384-dimensional. Warn about this before a first bulk ingest; after the model is cached, runs are local and fast.

Reading failures

Every failure renders as an RFC 9457 application/problem+json envelope with type, status, detail, and often suggested_fix + code_actions[]. Honor the applicability marker: apply a machine_applicable fix, surface a maybe_incorrect one to the user instead of auto-applying. A 404 document-not-found from find-similar means the anchor was never ingested — ingest it first.

How the suite uses this

doc-set-planner's corpus-aware Plan and Reconcile steps (shipped as their own change, stacked on this skill) consume it in two places: at Plan time, search the corpus for existing coverage of each proposed member (surface strong hits as "extend instead of create?" decisions); at Reconcile time, offer find-similar results as candidate relationships[] targets. In both cases the recipe contract and the deterministic planner-check gate stay authoritative.