rag

A Go library for multi-modal Retrieval-Augmented Generation with graph-enhanced retrieval, part of the saige SDK.

What it does

saige/rag models documents as hierarchical structures (Document -> Section -> ContentVariant) where each section can have multiple modality representations (text, image, table, audio). It provides a pluggable pipeline for ingesting documents, generating embeddings, performing hybrid search (vector + BM25 + graph), reranking, and assembling context with citations.

When to use

• Ingesting documents (PDF, text, images) into a searchable vector store
• Building RAG pipelines with multi-modal content support
• Combining vector search with knowledge graph traversal
• Evaluating retrieval and generation quality
• Deduplicating documents by content fingerprint

Usage

Install

go get github.com/urmzd/saige

Create a pipeline

import (
    "github.com/urmzd/saige/rag"
    "github.com/urmzd/saige/rag/pgstore"
    "github.com/urmzd/saige/rag/types"
    "github.com/urmzd/saige/postgres"
)

// Connect to PostgreSQL 18 (requires the pgvector and pg_search extensions).
pool, _ := postgres.NewPool(ctx, postgres.Config{URL: "postgres://localhost:5432/mydb"})
postgres.RunMigrations(ctx, pool, postgres.MigrationOptions{})

store, _ := pgstore.New(pgstore.Config{Pool: pool})
pipe, err := rag.New(rag.Config{Store: store},
    rag.WithContentExtractor(myExtractor),
    rag.WithEmbedders(myEmbedderRegistry),
    rag.WithRecursiveChunker(512, 50),
    rag.WithBM25(nil),
    rag.WithMMR(0.7),
)

Ingest a document

result, err := pipe.Ingest(ctx, &types.RawDocument{
    SourceURI: "https://example.com/paper.pdf",
    MIMEType:  "application/pdf",
    Data:      pdfBytes,
})

Search

results, err := pipe.Search(ctx, "attention mechanism", types.WithLimit(5), types.WithContextAssembly(4096))
fmt.Println(results.Context.Prompt) // context with citations

With knowledge graph integration

pipe, err := rag.New(rag.Config{Store: store},
    rag.WithContentExtractor(extractor),
    rag.WithGraph(kgGraph),  // enables entity extraction + graph retrieval
)

Key interfaces

Configuration options

Search options: types.WithCandidatePool(k) sets how many fused candidates reach the reranker, and types.WithMinScore(s) is a threshold in [0, 1] on the normalized fused score (1.0 keeps only hits ranked first by every retriever), applied before reranking.

Evaluation

import "github.com/urmzd/saige/rag/eval"

// Retrieval metrics (pure functions).
precision := eval.ContextPrecision(hits, relevantUUIDs)
recall := eval.ContextRecall(hits, relevantUUIDs)
ndcg := eval.NDCG(hits, relevantUUIDs, 10)
mrr := eval.MRR(hits, relevantUUIDs)

// Generation metrics (require LLM).
faith, _, _ := eval.Faithfulness(ctx, response, contextText, llm)
correctness, _ := eval.AnswerCorrectness(ctx, response, groundTruth, llm)

Agent tool bindings

import "github.com/urmzd/saige/rag/tool"

tools := tool.NewTools(pipeline)
// rag_search, rag_lookup, rag_update, rag_delete, rag_reconstruct

CLI

saige knowledge ingest --db "$SAIGE_RAG_DB" --file paper.pdf --mime application/pdf
saige knowledge sync --db "$SAIGE_RAG_DB" --dir docs --include '**/*.md' --exclude 'drafts/**' --prune
saige knowledge search --db "$SAIGE_RAG_DB" --query "attention mechanism"
saige knowledge show --db "$SAIGE_RAG_DB" --uuid <variant-uuid>
saige knowledge delete --db "$SAIGE_RAG_DB" --uuid <doc-uuid>