gnosis-mcp

# Gnosis MCP

> Zero-config MCP server that makes your markdown docs searchable by AI agents.
> SQLite default, PostgreSQL optional. Works with any MCP client — Claude Code, Claude Desktop, Cursor, Zed, opencode, Windsurf, VS Code, Cline.

## What it does

Gnosis MCP loads documentation files into a database, chunks them by headings (H2/H3/H4, never splitting code blocks or tables), and exposes them as MCP tools and resources. AI agents call `search_docs` to find relevant documentation instead of guessing or reading entire files. Supports `.md`, `.txt`, `.ipynb`, `.toml`, `.csv`, `.json` (stdlib only) + optional `.rst` (`[rst]` extra) and `.pdf` (`[pdf]` extra). Use `--watch` to auto-re-ingest when files change. Crawl documentation from any website with `gnosis-mcp crawl <url>` (requires `[web]` extra). Ingest git commit history as searchable context with `gnosis-mcp ingest-git <repo>` — zero new dependencies.

## Quick Start

```bash
pip install gnosis-mcp
gnosis-mcp ingest ./docs/       # auto-creates SQLite database + loads docs
gnosis-mcp setup --write        # wire every MCP client detected here, at the right path
gnosis-mcp doctor               # DB + wiring + whether anything has actually called it
gnosis-mcp serve                # start MCP server (stdio default, or --transport streamable-http)

# Web crawl (optional)
pip install gnosis-mcp[web]
gnosis-mcp crawl https://docs.example.com/ --sitemap   # crawl docs from the web

# Git history (no extra deps)
gnosis-mcp ingest-git .                                 # ingest commit history as searchable docs
gnosis-mcp ingest-git . --since 6m --include "src/*"    # filtered + time-limited
```

`gnosis-mcp setup` replaces the hand-copied snippet below for ten clients (Claude Code, DeepSeek
Harness, Codex CLI, Gemini CLI, Cursor, VS Code, Windsurf, Cline, Zed, and a generic stdio entry).
It resolves the absolute command path for *this* machine, writes each entry inside a
marker-delimited block that a re-run rewrites in place, and where the client does not forward the
server's MCP `instructions` field it also installs a short rule into that client's always-loaded
instruction file. Nothing is written without `--write`. `gnosis-mcp doctor` then reports the two
failures a config file cannot show: a wired path that no longer exists, and a client that is wired
but has never actually called the server.

Add to your client's MCP config (e.g. `.mcp.json` at your project root for Claude Code):

```json
{
  "mcpServers": {
    "docs": {
      "command": "gnosis-mcp",
      "args": ["serve"]
    }
  }
}
```

## Tools

Six read tools are always advertised. The three write tools appear in `tools/list` only when the server runs with `GNOSIS_MCP_WRITABLE=true`, so a read-only client is never handed a tool it cannot call.

### Read Tools (always available)

#### search_docs(query: str, category?: str, limit?: int, query_embedding?: list[float], rerank?: bool) -> JSON
Search documentation using keyword (FTS5/tsvector) or hybrid semantic+keyword search.
When `query_embedding` is provided, hybrid mode merges BM25 and cosine scores via Reciprocal Rank Fusion (tune via `GNOSIS_MCP_RRF_K`, default 60).
When `rerank=true` (or `GNOSIS_MCP_RERANK_ENABLED=true`), a cross-encoder re-scores the top-N candidates (requires the `[reranking]` extra).
Returns: `[{"file_path", "title", "content_preview", "score", "highlight", "rerank_score"?}]`
The `highlight` field contains matched terms in `<mark>` tags (FTS5 snippet on SQLite, ts_headline on PostgreSQL).

#### get_doc(path: str, max_length?: int) -> JSON
Retrieve full document by file path. Reassembles chunks in order.
Returns: `{"title", "content", "category", "audience", "tags"}`

#### search_git_history(query: str, author?: str, since?: str, until?: str, file_path?: str, limit?: int) -> JSON
Search indexed git commit history (the `git-history` category) by message, author, date range, or file.
Returns: same shape as `search_docs`.

#### get_related(path: str, depth?: int, relation_type?: str, include_titles?: bool) -> JSON
Find related documents via bidirectional link graph. `depth` goes up to 3 (multi-hop); `relation_type` filters to e.g. `relates_to`, `content_link`, `git_co_change`.
Returns: `[{"related_path", "relation_type", "direction"}]`

#### get_context(topic?: str, limit?: int, category?: str) -> JSON
Usage-weighted context summary. Without `topic`: most-accessed documents plus corpus stats. With `topic`: search results enriched with access counts.
Returns: `{"docs", "stats"}`

#### get_graph_stats(category?: str) -> JSON
Knowledge graph topology: orphans, hubs, relation distribution, node/edge counts.
Returns: `{"total_docs", "total_edges", "relation_types", "hubs", "orphans"}`

### Write Tools (requires GNOSIS_MCP_WRITABLE=true)

#### upsert_doc(path: str, content: str, title?: str, category?: str, audience?: str, tags?: list, embeddings?: list[list[float]]) -> JSON
Insert or replace a document. Auto-splits into chunks at paragraph boundaries.
Optional `embeddings` parameter accepts pre-computed vectors (one per chunk).
Returns: `{"path", "chunks", "action": "upserted"}`

#### delete_doc(path: str) -> JSON
Delete a document and all its chunks + related links.
Returns: `{"path", "chunks_deleted", "links_deleted", "action": "deleted"}`

#### update_metadata(path: str, title?: str, category?: str, audience?: str, tags?: list) -> JSON
Update metadata on all chunks of a document. Only provided fields change.
Returns: `{"path", "chunks_updated", "action": "metadata_updated"}`

## Re-indexing a re-organized corpus

When your knowledge folder changes significantly (files moved or deleted), use one of:

- `gnosis-mcp ingest ./docs --prune` — ingest new/changed, remove DB chunks whose source file no longer exists
- `gnosis-mcp ingest ./docs --wipe` — full reset before re-ingest (nuclear)
- `gnosis-mcp prune ./docs --dry-run` — preview what would be deleted without touching anything
- `gnosis-mcp prune ./docs` — delete stale only, no ingest

Pruning is scoped to the given root; crawled URLs and generated documents (git history) are preserved unless `--include-crawled` / `--include-generated` is passed.

## Resources

- `gnosis://docs` -- list all documents (path, title, category, chunk count)
- `gnosis://docs/{path}` -- read document content by path
- `gnosis://categories` -- list categories with doc counts

## REST API (v0.10.0+)

Enable with `--rest` flag or `GNOSIS_MCP_REST=true`. Runs alongside MCP on the same HTTP port. Endpoints: `GET /health`, `GET /api/search?q=&limit=&category=`, `GET /api/docs/{path}`, `GET /api/docs/{path}/related`, `GET /api/categories`. Optional CORS (`GNOSIS_MCP_CORS_ORIGINS`) and API key auth (`GNOSIS_MCP_API_KEY`). No new dependencies — uses Starlette (already bundled with mcp>=1.20).

## Backends

| | SQLite (default) | SQLite + embeddings | PostgreSQL |
|---|---|---|---|
| Install | `pip install gnosis-mcp` | `pip install gnosis-mcp[embeddings]` | `pip install gnosis-mcp[postgres]` |
| Config | Nothing | Nothing | Set `DATABASE_URL` |
| Search | FTS5 keyword (BM25) | Hybrid keyword + semantic (RRF) | tsvector + pgvector hybrid |

Auto-detection: `GNOSIS_MCP_DATABASE_URL` set to `postgresql://...` -> PostgreSQL. Not set -> SQLite at `~/.local/share/gnosis-mcp/docs.db`. Override: `GNOSIS_MCP_BACKEND=sqlite|postgres`.

Ingestion extracts `relates_to` from frontmatter (comma-separated or YAML list) and populates the links table for `get_related` queries.

Local embeddings (`[embeddings]` extra): ONNX Runtime + tokenizers + sqlite-vec. Default model: MongoDB/mdbr-leaf-ir (23MB, Apache 2.0). No API key needed.

## Configuration

Set `GNOSIS_MCP_DATABASE_URL` (or `DATABASE_URL`) for PostgreSQL. Leave unset for SQLite. Optional: `GNOSIS_MCP_BACKEND`, `GNOSIS_MCP_SCHEMA`, `GNOSIS_MCP_CHUNKS_TABLE` (comma-separated for multi-table on PG), `GNOSIS_MCP_LINKS_TABLE`, `GNOSIS_MCP_SEARCH_FUNCTION`, `GNOSIS_MCP_EMBEDDING_DIM`, `GNOSIS_MCP_WRITABLE`, `GNOSIS_MCP_WEBHOOK_URL`, `GNOSIS_MCP_COL_*` for column names. Embedding: `GNOSIS_MCP_EMBED_PROVIDER` (openai/ollama/custom/local), `GNOSIS_MCP_EMBED_MODEL`, `GNOSIS_MCP_EMBED_DIM` (384, for local Matryoshka truncation), `GNOSIS_MCP_EMBED_API_KEY`, `GNOSIS_MCP_EMBED_URL`, `GNOSIS_MCP_EMBED_BATCH_SIZE`. Tuning: `GNOSIS_MCP_CONTENT_PREVIEW_CHARS`, `GNOSIS_MCP_CHUNK_SIZE`, `GNOSIS_MCP_SEARCH_LIMIT_MAX`, `GNOSIS_MCP_WEBHOOK_TIMEOUT`, `GNOSIS_MCP_TRANSPORT` (stdio/sse/streamable-http), `GNOSIS_MCP_HOST`, `GNOSIS_MCP_PORT`, `GNOSIS_MCP_LOG_LEVEL`.

## Database Schema

Chunks table: `(file_path, chunk_index, title, content, category, audience, tags, embedding, content_hash)`
Links table: `(source_path, target_path, relation_type)`

Tables are auto-created on first `serve` or `ingest`, so wiring up a client before your first ingest works. Run `gnosis-mcp init-db` to create them manually, or `--dry-run` to preview SQL.

## Editor Config Locations

| Editor | Config Path |
|--------|------------|
| Claude Code | `.mcp.json` at the project root |
| Claude Desktop | `claude_desktop_config.json` — `~/Library/Application Support/Claude/` (macOS), `%APPDATA%\Claude\` (Windows) |
| Cursor | `.cursor/mcp.json` |
| Zed | `settings.json` (uses `"context_servers"` key, not `"mcpServers"`) |
| opencode | `opencode.json` or `~/.config/opencode/opencode.json` (uses `"mcp"` key, command as an array) |
| VS Code (Copilot) | `.vscode/mcp.json` (uses `"servers"` key, not `"mcpServers"`) |
| Windsurf | `~/.codeium/windsurf/mcp_config.json` |
| JetBrains | Settings > Tools > AI Assistant > MCP Servers |
| Cline | Cline MCP settings panel |

## Performance

9,463 QPS on 100 docs (300 chunks), 471 QPS on 10,000 docs (30,000 chunks). p95 under 0.2 ms at 100 docs, under 6 ms at 10,000. End-to-end through the MCP stdio protocol: 8.7 ms mean, 13.0 ms p95 (v0.13.3 SDK 1.27 transport). 862 tests, 10 RAG eval cases (Hit@5 = 1.00, MRR = 1.00, Precision@5 = 0.75). Install size: ~23MB with `[embeddings]` (ONNX model), ~5MB base. Reproduce with `gnosis-mcp eval`, `python tests/bench/bench_search.py`, `python tests/bench/bench_rag.py`, `python tests/bench/bench_mcp_e2e.py`.

## Documentation

- [README](README.md): Quick start, editor setup, backend comparison, configuration, performance
- [llms-full.txt](llms-full.txt): Complete reference in one file
- [llms-install.md](llms-install.md): Step-by-step install guide