Agent Architecture Decision Framework

Overview

Agent architecture decisions compound. A wrong early decision (e.g. single agent with tool-use when you need observability) becomes expensive to fix later. Make these four decisions explicitly, in order, before writing any code.

Decision 1: Workflow vs Agent

The question: Does your pipeline need to recover from mid-step failures, or is it always the same sequence?

Default recommendation: Agent, unless your pipeline is truly linear with no recovery needed. The ability to route back on failure (e.g. SQL validation failure → regenerate SQL) is almost always worth the small added complexity.

Decision 2: Single Agent vs Multi-Agent

The question: Do your components need independent state and memory, or can they share state?

Default recommendation: Single agent with specialized nodes. For a single-pipeline system, multi-agent adds complexity that rarely pays off. Use multi-agent when you genuinely need independent memory per component, or when components run concurrently with different LLM models and goals.

The overkill test: If your "multi-agent" system has one orchestrator directing all other agents sequentially, it's a single agent with extra steps. Flatten it.

Decision 3: Tool-Use vs Specialized Nodes

The question: Should one LLM decide which tools to use, or should each step be a dedicated node?

Default recommendation: Specialized nodes for any pipeline with known steps. Tool-use is better for open-ended tasks where you genuinely don't know what steps are needed (e.g. a research agent that might need 2 or 20 searches).

The sequence test: If you can write down the steps your pipeline takes before running it, use specialized nodes. If the steps only become clear at runtime, use tool-use.

Decision 4: Retrieval Strategy

The question: How does your agent get information from its data source?

For SQL-based retrieval, MUST implement these safety layers:

1. Syntax validation before execution (catch LLM SQL errors early)
2. Destructive keyword blocking (regex on stripped quoted content — block DROP, DELETE, UPDATE, INSERT, ALTER, TRUNCATE)
3. Read-only connection (defense in depth even after validation)
4. Row cap at Python level, not just in prompt (LLM may omit LIMIT clause)

Decision 5: Where to Put Intelligence

For each step in your pipeline, decide: LLM or deterministic?

Principle: Every deterministic component you add reduces variance and improves debuggability. Never use an LLM for something a simple function can do.

Step-by-Step Architecture Decision Process

When starting a new project:

1. Write down the pipeline steps in order (even if approximate)
2. Apply Decision 1 (workflow vs agent) — does any step need to retry an earlier step?
3. Apply Decision 2 (single vs multi) — do any components need independent state?
4. Apply Decision 3 (tool-use vs nodes) — is the step sequence known in advance?
5. Apply Decision 4 (retrieval strategy) — what is the data source and query type?
6. Apply Decision 5 (LLM vs deterministic) — for each step, which is appropriate?
7. Draw the graph — nodes, edges, routing conditions, loop guards
8. Identify the blast radius of each LLM node (what downstream nodes consume its output?)
9. Write the loop guards — every cycle in the graph needs a hard exit condition

Architecture Evolution Warning

Systems evolve. The architecture that was right at design time may not be right after:

• New query types require new routing
• A simple node grows into a multi-step process
• An LLM node turns out to need deterministic behavior

Watch for these signals that architecture needs revisiting:

• A node's prompt is over 100 lines
• A single node is doing more than one conceptually distinct thing
• Debugging a failure requires tracing through more than 3 nodes to find root cause
• Adding a new feature requires changing 4+ files

When these appear, revisit the relevant decision rather than adding more complexity on top.