Onboard a repository to HyperMnesia + Serena

Six steps. Five are mechanical; step 3 is the one that decides whether any of this is worth it, so budget your attention there.

The end state: before the agent edits src/api/routes.py it already knows the rules for that file (no search, no model), it can find the doc that explains the auth flow, it remembers what you decided last month — and it asks a language server, not a vector index, where foo is defined.

0. Check the prerequisites

psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -c 'select 1'                  # store reachable
psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -c 'select count(*) from components'   # schema loaded
curl -s localhost:11434/api/tags | head -c 80        # embedder alive (Ollama; TEI: /health)

-v ON_ERROR_STOP=1 on every psql call here, including the ones below: without it psql prints the error to stderr and still exits 0, so a failed statement reads as an empty result — the exact shape of failure this whole system exists to avoid.

If the schema is missing: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f sql/schema.sql -f sql/schema_mem.sql. If there is no store at all, ./hm init brings up the Docker stack (Postgres + TEI, every published port bound to 127.0.0.1, a generated password in deploy/docker/.env), loads both schemas, and prints the DATABASE_URL / EMBED_BACKEND lines to export.

Pick the repo tag now and use it everywhere — it is the scoping key for the whole tier and is matched as an exact string, case included. Ingest rejects a tag outside [A-Za-z0-9._-]; the hook strips every other character from HM_REPO (or from the cwd basename, when HM_REPO is unset), and the MCP server refuses such a name, and ALL, outright and scopes the session to nothing. Neither ever changes the case. So ingest under the tag you will set as HM_REPO, and set HM_REPO in both places that resolve — the hook env (step 5) and the MCP server env (step 4). A repo at ~/code/MyRepo ingested as myrepo resolves only with HM_REPO=myrepo set; without it the fallback is the directory basename MyRepo, which matches nothing, forever, quietly. Both sides now say so: the MCP server prefixes its answer with a (!) SCOPE: line when it guessed the name from the directory or had to refuse it, and the hook announces once per episode that nothing is mapped under the scope, listing the scopes that exist. The server's note appears on get_project_map, get_document and search_docs — the three whose answers the scope decides. get_constraints and locate do not carry it, so a wrong scope there still looks like a repo with no rules.

1. Ingest the docs

./hm ingest /path/to/repo <repo-tag>

Prefer the wrapper. It shells the same scripts as the manual steps, in the order that works, and it does two things whose failure mode is silent when you drive them by hand: when the scope already holds documents it takes the known-hashes snapshot from the database it is about to write to (otherwise a re-ingest deletes the scope's documents and every embedding with them), and it creates the HNSW index after the first bulk embed rather than before. It then runs EMBED_REPO=<repo-tag> ingest/embed_chunks.py and finishes with ci/doctor.py, so steps 1 and 2 are one command. It needs DATABASE_URL and psql on PATH, and refuses a scope outside [A-Za-z0-9._-] before it touches the database.

Do it by hand when the store is reachable only from somewhere else — ingest_repo.py never connects to a database, so it runs where the files are and the SQL is applied where the store is:

python ingest/ingest_repo.py /path/to/repo <repo-tag> out.sql
psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f out.sql

Markdown only, chunked by heading. Things to know before you trust the result:

• A git-ignored corpus needs --walk. git ls-files is the default enumeration, and it returns nothing for a directory that is inside a repo but .gitignored — working notes and scratch docs, typically. The ingester refuses to write an empty corpus rather than emitting one that would delete what is already stored; pass --walk to index the tree directly. ./hm ingest has no --walk, so that corpus has to go through the manual path.
• A full re-ingest deletes and re-inserts the repo's documents. A constraint that names its source by source_repo/source_path (sql/migrate_constraint_sources.sql) is re-linked to the new row automatically; one that sets only source_doc_id loses the link (the FK is ON DELETE SET NULL), so name sources by path in the seed. A full re-ingest also drops every chunk (that FK cascades) and chunks carry the embeddings — so a re-ingest for one edited file re-embeds the whole corpus.

• Re-ingesting by hand? Use --known-hashes (./hm ingest already does this for you). Hand the ingester what the DB already holds and only new and changed files are rewritten; unchanged documents keep their chunks, their embeddings, and their source_doc_id links. The ingester still needs no DB connection — the hashes arrive as a file, so this works over kubectl exec or a tunnel like everything else here:

  psql "$DATABASE_URL" -tAX -v ON_ERROR_STOP=1 -F$'\t' \
    -c "SELECT path, content_hash FROM documents WHERE repo='<repo-tag>'" > known.tsv
  python ingest/ingest_repo.py /path/to/repo <repo-tag> out.sql --known-hashes known.tsv
  psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f out.sql

  Take the snapshot from the database you are about to load into. The SQL checks that the repo still holds as many documents as the snapshot describes and aborts if it doesn't — a snapshot from another environment would silently skip documents that database never had, leaving a corpus with holes that reads as "search finds nothing".

• A file git lists but that is not on disk is treated as deleted, and said so on stderr ("listed by git but not on disk"). The usual cause is a working-tree deletion that has not been staged: git ls-files reads the index, so it still names the file. Its stored document is pruned, which is what you want — but if you did not mean to delete it, restore it before ingesting rather than after.

2. Fill in the embeddings

(./hm ingest already ran this for the scope it ingested; do it separately for the manual path.)

EMBED_REPO=<repo-tag> python ingest/embed_chunks.py   # only rows where embedding IS NULL

Resumable — safe to interrupt and re-run. There is no "re-embed what already has a vector" switch: the selection is always embedding IS NULL, narrowed by the environment below. To recompute existing vectors you clear them first, as the model-change recipe does. What it reads from the environment is EMBED_REPO (comma-separated scopes; all repos when unset — worth setting, since the chunk text is the request body sent to the embedder), EMBED_BATCH (16) and SHARD_N/SHARD_I to split one pass across replicas, plus the usual EMBED_BACKEND / OLLAMA_URL / TEI_URL / EMBED_MODEL.

Rows are stamped with embedding_model, so changing embedder means clearing the differently stamped rows yourself and re-running:

psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -c \
  "UPDATE chunks c SET embedding=NULL FROM documents d
    WHERE d.id=c.document_id AND d.repo='<repo-tag>'
      AND c.embedding_model IS DISTINCT FROM '<new-model>'"
EMBED_MODEL=<new-model> EMBED_REPO=<repo-tag> python ingest/embed_chunks.py

EMBED_MODEL selects the model only on the Ollama backend. With EMBED_BACKEND=tei the request carries no model name at all — the model is whichever one that TEI instance was started with, so switching there means starting a different TEI, and EMBED_MODEL then only controls the string stamped into embedding_model. Stamping a name the server is not serving is worse than not stamping one, because the tripwire that detects a mixed store is exactly that column.

chunks.embedding is vector(1024) (sql/schema.sql), so only another 1024-dim model is a drop-in; a different dimension needs a schema change, not a re-embed.

3. Author the Tier 0/1 map — the part that matters

Everything else is plumbing. This is the payload: a hand-written component graph with the rules that apply to each area. Start from examples/seed_example.sql.

Guidance that comes from getting it wrong:

• Write rules that change behaviour, not descriptions. "No component other than the data layer may import the Postgres driver" is a rule. "The API layer handles HTTP" is a label; it costs tokens on every edit and changes nothing.
• must is injected before every edit; should is not. Only what genuinely blocks a change belongs at must, or the injection becomes noise people learn to skim.
• key_paths globs decide everything. ** crosses / (and **/ also matches zero segments, so src/**/*.py matches src/main.py), * and ? do not cross /, and matching is exact-beats-longest-literal-prefix-beats-priority. Dot-prefixed paths (.gitlab-ci.yml, .claude/**) are ordinary paths here: unlike shell globbing, * and ** match a leading dot, so no special-casing is needed. Only a leading ./ is stripped from the path being resolved.
• Relationships pull one hop, in both directions. Declaring api depends_on db means editing an API file also surfaces the data layer's rules — and editing a data-layer file surfaces the API's, because the expansion follows the edge either way regardless of kind. That is usually what you want, and it is also how a sloppy graph floods the agent with irrelevant rules.

Apply it and confirm the map answers for a real file:

psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -f your_seed.sql
printf '{"tool_name":"Edit","cwd":"/path/to/repo","tool_input":{"file_path":"/path/to/repo/src/api/routes.py"}}' \
  | HM_REPO=<repo-tag> python hooks/arch_invariants.py

Empty output means no must was selected for that path — either the globs match nothing, or what they match carries only should/info. Fix the globs now, because a silent Tier 1 is worse than none: it looks like "no rules apply". Faults are announced instead of injected, but only on their first occurrence: the flip is a marker file hypermnesia-<fault> (store-down, graph-unparseable, no-repo-<tag>) in ~/.HyperMnesia/state/ -- in ~/.claude/ on a machine where ./hm install has not run yet -- so delete it to see the notice again.

4. Wire both MCP servers

HyperMnesia answers "what are the rules / where is the doc / what do I know". An LSP server answers "where is this symbol". Do not make either do the other's job — see docs/ARCHITECTURE.md.

{
  "mcpServers": {
    "hypermnesia": {
      "command": "/home/you/.HyperMnesia/current/bin/hypermnesia-mcp",
      "env": {
        "HM_REPO": "<repo-tag>",
        "DATABASE_URL": "postgresql://hm:pass@localhost:5432/hypermnesia",
        "EMBED_BACKEND": "ollama",
        "HM_SEARCH": "/home/you/.HyperMnesia/current/ingest/search.py",
        "HM_MEM_OPS": "/home/you/.HyperMnesia/current/ingest/mem_ops.py"
      }
    },
    "serena": {
      "command": "serena",
      "args": ["start-mcp-server", "--context", "claude-code", "--project", "/path/to/repo"]
    }
  }
}

5. Register the hooks

In the project's .claude/settings.json (see hooks/README.md for the full set and for Codex CLI, which uses the same contract):

{"hooks": {
  "PreToolUse":      [{"matcher": "Edit|Write|MultiEdit", "hooks": [{"type": "command", "command": "HM_REPO=<repo-tag> python3 /home/you/.HyperMnesia/current/hooks/arch_invariants.py", "timeout": 20}]}],
  "SessionStart":    [{"hooks": [{"type": "command", "command": "python3 /home/you/.HyperMnesia/current/hooks/mem_profile.py"}]}],
  "UserPromptSubmit":[{"hooks": [{"type": "command", "command": "python3 /home/you/.HyperMnesia/current/hooks/mem_recall.py"}]}]
}}

The constraint hook injects context and must not return a permissionDecision — emitting allow there silently auto-approves every edit that happens to have a must.

6. Verify, then walk away

HM_REPO=<repo-tag> ./hm doctor            # unbuilt ANN index, unembedded chunks, unmapped scope
python tests/test_hook_contract.py        # hook I/O shape (hooks are fail-open = silent when broken)
python eval/mem_probes.py                 # staleness / abstention / temporal / recall
python ci/freshness.py /path/to/repo <repo-tag>   # globs that match no file; docs behind HEAD
python ingest/search.py "how does X work" 5 <repo-tag>   # a real question you know the answer to

freshness.py exits 1 on any map orphan, and also when the scope has no components at all — the case where every other count would read as a clean zero. Everything it does without --mark is a SELECT; --mark writes (documents.status='stale'). freshness.py is the one to schedule. The map is the asset and it rots silently: when a directory moves and its glob does not, Tier 1 stops resolving and says "no rules" with total confidence. Run it in the repo's CI.

Adapting to your deployment

Only how the scripts reach the store changes; the six steps do not.

Whatever the transport, keep every hook fail-open: a memory system that can block an edit is worse than no memory system. But make an outage visible at session start — otherwise "the store is down" is indistinguishable from "nothing relevant is stored", and a machine can run with zero memory for weeks without anyone noticing.