Agentic RAG (evidence-gated Q&A)

Overview

Implementation playbook for the agentic RAG support-desk architecture (source: Daily Dose of DS "Jev-powered Agentic RAG" walkthrough). The system retrieves evidence, asks a decision model whether the evidence is sufficient, answers with citations when it is, and escalates to a human ticket when it is not. Human replies feed back into the index.

Five slots, stack-agnostic:

Architecture selector (when to use what)

Context separation (Conversation vs. Memory vs. RAG)

Keep the three context sources distinct in storage metadata and in the generator system prompt:

Iron rules

1. Never call the generation model when the gate fails. Escalation is a first-class output, not an error.
2. Thresholds live in a DB/config table, separate for text and image candidates; tuning must never require a deploy.
3. Log every gate decision (candidates, scores, verdict) to decision_log before answering or escalating.
4. Index each page twice: ~220-word text chunks AND a rendered page PNG, both tagged with the page number. Merge candidates by page.
5. Answers must cite page numbers and return thumbnails/excerpts when the channel supports it. The generator answers only from supplied evidence.
6. One wider retry max before ticketing: widen top-k/filters once, never loop.
7. Human actions stay human: reps write replies, customer receives email. "Save to knowledge" re-indexes the Q&A pair as a new chunk.
8. Tenant and user isolation via RLS / authenticated principal (ClaimsPrincipal / JWT): never accept user_id or session_id from the request body, and escape or parameterize all metadata filters.
9. No silent rerank or gate fallback: inspect stream status events (RERANKING_FAILED, SUMMARIZATION_FAILED) and verify the rerank/gate stage actually ran; enforce max_results cut even when reranking is off.
10. End-to-end stream deadline and reasoning token budget: HttpClient.Timeout only covers response headers on streamed replies; link a request-wide deadline across history load + stream read + turn store, and size gen_token_budget (e.g. >=2048) so reasoning models do not exhaust budget on thinking tokens.

Phase route

Acceptance criteria

• Query with sufficient evidence: answer + page citations + decision_log row.
• Query with insufficient evidence: single wider retry, then ticket with candidate scores visible to the rep; generation model never called.
• Rep reply reaches customer by email; save-to-knowledge makes the same question retrievable.
• Threshold change in DB alters gate behavior without a deploy.

Non-goals

• No auto-submit/auto-send to end users without rep confirmation.
• No figure-level extraction (full-page PNG only; vector diagrams lose captions otherwise).
• No unbounded retry loops, no answer on sub-threshold evidence.

Integration notes

• data-analyst/research skills can reuse the gate pattern for evidence sufficiency before answering.
• On this bundle host, extensions/voicebox (tts) + asr can add voice to the desk frontend; optional, not part of the core pipeline.