markdown-memory - agent instructions

Local MCP server for AST-aware Markdown indexing: SQLite with FTS5 (BM25), sqlite-vec section and passage vectors, EmbeddingGemma-300m (default) or bge-small embeddings, fused with Reciprocal Rank Fusion. CLAUDE.md is the full developer guide (layout, architecture rules, regression policy); this file is the short form for agents that do not read it.

Commands

uv sync                                     # set up
uv run src/markdown_memory/server.py        # run the MCP server (stdio)
uv run ruff check . && uv run ruff format . # lint, format
uv run mypy --strict src/                   # type check (zero errors required)
uv run pytest                               # tests
uv run python scripts/eval_retrieval.py     # retrieval gate: Top-1 >= 80%, Top-5 >= 90%, identifiers 100%
uv run python scripts/live_test.py          # end-to-end over stdio
scripts/check.sh                            # all of the above except the eval, fail-fast

Non-negotiable rules

• Python 3.11+, mypy --strict clean, no untyped defs, no unconstrained Any.
• Domain models are @dataclass(slots=True, frozen=True).
• SQLite: WAL mode, writes only inside with db.transaction():.
• Never write to stdout in src/ (it is the JSON-RPC channel); log to sys.stderr.
• Any change to ranking, chunking or embedding must pass scripts/eval_retrieval.py. Never tune on the held_out queries.

Reading Markdown documentation (when the markdown-memory MCP server is mounted)

Never dump whole files. Do not Read/read_file/cat a Markdown document longer than ~150 lines into context unless the user explicitly asks for the whole file. Fetch the one section you need instead; it is typically 10-50x cheaper.

Pass cwd - your working directory, as an absolute path - on every call. In another git worktree of the repository the server was started for, answers then come from that worktree's own copy of the docs, indexed into its own database the first time it is named; anywhere else it changes nothing. index_status.root names the documentation root that answered: when it is not the tree you are working in, you are reading another checkout's docs. Every file_path in an answer is relative to that root (absolute when another worktree answered): pass it back verbatim, with the same cwd.

Work in this order:

1. search_docs(query, limit=5) - start here. Returns {"results": [...], "keyword_match": ..., "index_status": {...}}, plus a keyword_message whenever keyword_match is not "matched". The first hit carries the full section content with its breadcrumb (heading_path), line range and token estimate - or, marked excerpt: true, only the passage that matched and its neighbours, verbatim, with lines naming them and tokens still what the whole section costs: when the excerpt is not enough, read_section(file_path, heading_path) returns the full section. The rest are pointers - file_path, heading_path, lines, tokens and, when a passage won the vector ranking, matched_passage: how that passage begins. Use exact identifiers verbatim (--dry-run, HELIOS_BATCH, ENOSPC): they are matched by keyword at 100% Top-1. Plain-language questions work too, but the first hit is not always the answer (Top-1 ~88%; Top-5 is ~97% reliable): when it is not, read the pointers' matched_passage and fetch the one that fits with read_section, passing its heading_path verbatim - for a (Part n) the base path returns every part. A (Part n) pointer also carries part_preview, how that part begins, so parts of one split section can be told apart before reading any of them. keyword_match says whether keyword search found the query's terms; when it is "no_match", no indexed section contains them: a query of identifiers alone then returns no results, and any other query's hits are only semantic neighbours. Report such an identifier as not in the indexed documentation rather than answering from neighbours - unless index_status.indexing is true and coverage "unknown": then it may only not be indexed yet, so search again once indexing is false. index_status.gitignore is what git said when a run of the root last finished: applied, off, no_repository, unavailable (git could not be asked, so ignored files may be in the index; the message says how to see why) or unknown - no run has recorded it yet, which is not a failure and needs nothing from you. When index_status.coverage is "unknown", the documentation you just searched is missing files or was never indexed end to end - say so rather than concluding the docs do not cover it. When index_status.changed_files is non-zero, that many indexed documents could not be confirmed to be what was indexed - edited, unreadable or deleted - so a hit may quote text that is no longer there; re-run index_directory before relying on it. The count is best-effort: files created since the last run are not in it, and an edit that puts a file's modification time back is not seen, so a zero is "nothing detected", not "everything verified".
2. get_document_outline(file_path) - only when you need the structure of a document: the heading tree with line spans and per-section token estimates, at a few hundred tokens. Use it to choose a section, or to find sibling sections of a search hit.
3. read_section(file_path, heading_path) - fetch exactly one section, verbatim. heading_path is the breadcrumb Root > Child > Subchild as shown by the outline or by search_docs; a bare heading title works when it is unambiguous. It returns the section's own text only - pass include_subsections=true for a parent heading whose content lives in its children. Oversized sections appear as Path (Part n); reading the base path returns all parts reassembled.

Supporting tools: list_documents(directory="") returns {"documents": [...], "index_status": {...}} - what is indexed, and whether anything vouches for it; index_directory(directory=None) (re)indexes. The stdio server keeps its own docs root indexed in the background (from the first search, and whenever a search sees change), so call it only when index_status.message asks for it - never while index_status.indexing is true. It is incremental (SHA-256 per file), so re-running is cheap. file_path may be absolute, relative to the docs root, or any unique path suffix. An error from read_section lists the valid heading paths - retry with one of them rather than falling back to reading the file.