Fan-Out / Fan-In (Workers + Orchestrator)

Also known as "The Council & The Judge."

Overview

Run the same problem through N independent worker agents, each in its own fresh context, then have one stronger orchestrator agent review, deduplicate, rank, and arbitrate their outputs into a single consolidated answer.

It is the parallelization (sectioning + voting) and orchestrator-workers patterns combined with an evaluator step, applied as a repeatable working method rather than a one-off.

Core principle: an LLM is a stochastic generator with context-sensitive fidelity. One sample is one draw from a distribution. Fanning out takes several independent draws; fanning in lets a capable judge keep what is corroborated, demote what is weak, and discard what is wrong.

When to use

Use when any of these are true:

• Re-running the same prompt gives a different set or ordering of findings each time (a, b, c, d, e → a, b, c, e, g → a, b, e, h, j …). The result is unstable.
• The problem is open-ended with no single right path (research, planning, architecture, audits, "what did we miss?").
• The input is large — long documents, many files, big logs — so one agent's context fills up and fidelity degrades (see the figure below).
• The cost of a missed or wrong point is high (client deliverables, security, migrations, anything hard to reverse).
• You want a defensible answer: "five independent passes agreed on X" is stronger than "one pass said X."

When NOT to use:

• Simple, deterministic, or single-answer tasks (a lookup, a rename, a format).
• Tightly sequential work where each step depends on the previous one — there is nothing to parallelize. Fan-out shines on independent subtasks/perspectives.
• Throwaway low-stakes work where the extra token cost is not justified. Fan-out trades tokens and latency for reliability (multi-agent setups can use ~15× the tokens of a single chat) — spend it only when the answer is worth it.

Why it works — the two problems it solves

1. LLMs are stochastic

Identical prompts produce different outputs across runs, even at low temperature — Anthropic notes agents are "non-deterministic between runs, even with identical prompts." So a single pass is one sample from a distribution of plausible answers. Important points appear in some runs and not others.

Fan-out = draw several independent samples. A point that surfaces in 4 of 5 independent workers is far more likely to be real than one that appears once. The orchestrator turns these samples into a ranked consensus instead of a coin flip.

This is the voting variant of parallelization: run the same task multiple times to get diverse outputs, then aggregate.

2. Fidelity degrades as the context window fills

Model accuracy is not flat across context length. As the used window grows, attention spreads thin and models attend less reliably to the middle of long inputs ("lost in the middle" / "context rot"). A single agent asked to hold a huge problem and reason over it operates in the degraded regime.

Image: Performance vs. context window

Illustrative — synthesised from published trends (see references/evidence.md).

Fan-out = keep every worker in the short-context, high-fidelity zone. Each worker sees only its slice (or the whole problem with a tight remit) and returns a compact, distilled result. The orchestrator only ever reads those short summaries — never the raw flood. Anthropic: subagents "operate in parallel with their own context windows … condensing the most important tokens for the lead agent."

Anthropic found a multi-agent system (Opus lead + Sonnet subagents) outperformed a single-agent Opus by 90.2% on their internal research eval, and that token usage alone explained ~80% of performance variance — distributing work across separate context windows adds usable capacity. (evidence)

Model assignment — cheap workers, smart judge

This mirrors routing easy work to cheaper models and hard work to capable ones, and Anthropic's own Opus-lead / Sonnet-subagent architecture. You spend the expensive model's tokens only on the judgement step.

The method

flowchart TD
    Q[Problem / question] --> P[Frame: define remit, output format,<br/>and what 'done' looks like]
    P --> W1[Worker 1<br/>fresh context]
    P --> W2[Worker 2<br/>fresh context]
    P --> W3[Worker 3<br/>fresh context]
    P --> Wn[Worker N<br/>fresh context]
    W1 --> O[Orchestrator review<br/>capable model]
    W2 --> O
    W3 --> O
    Wn --> O
    O --> R[Consolidated answer:<br/>ranked, deduped, conflicts resolved]
    R --> D{Good enough?}
    D -- no, gaps remain --> P
    D -- yes --> Done[Deliver]

1. Frame the problem (once). Write a single, explicit brief: the objective, the output format every worker must return, the boundaries, and what "done" looks like. Vague briefs make workers duplicate each other or drift — give each a clear remit. (Anthropic: "teach the orchestrator how to delegate.")
2. Fan out to N independent workers (typically 3–5). Each runs in a fresh context with the same brief (voting) or a partitioned slice (sectioning). They must not see each other's work — independence is the whole point. Scale N to difficulty: simple → 3, complex → 5+.
3. Each worker returns a compact, structured result. Findings as a ranked list with a one-line rationale each, not a wall of prose. Short outputs keep the orchestrator in the high-fidelity zone.
4. Fan in to one orchestrator. The capable model reads all N outputs and produces a single consolidated answer by applying the rubric below.
5. Iterate if needed. If the orchestrator flags gaps or unresolved conflicts, re-fan-out on just those open questions. Stop when consensus is stable.

Orchestrator rubric (the fan-in judgement)

The orchestrator classifies every distinct point raised by any worker:

• Corroborated / high-priority — raised independently by multiple workers, or raised once with strong, checkable evidence. Keep and rank at the top.
• Plausible / lower-priority — raised by one worker, reasonable but unconfirmed. Keep, but mark as lower confidence; consider a targeted re-check.
• Wrong / drop — contradicted by other workers or by checkable facts, or based on a faulty assumption. Discard, and say why (so the reasoning is auditable).
• Conflict — workers disagree. The orchestrator arbitrates with reasons, or spawns a focused tie-breaker worker.

Output: a single ranked list (high → low priority) plus an explicit dropped-with-reasons section. Frequency across workers is a signal, not a vote-to-win — a lone worker with a verifiable fact beats three that hand-wave.

Worked origin — how this method was arrived at

The method came directly from watching the two failure modes above bite on real work:

1. Observed stochasticity. Asking one model the same hard question repeatedly returned overlapping but non-identical answer sets — a,b,c,d,e, then a,b,c,e,g, then a,b,e,h,j. The stable core (a, b) was trustworthy; the tail was luck of the draw. Conclusion: sample more than once and look at agreement.
2. Observed context degradation. Loading a whole large problem into one agent produced shallower, more error-prone reasoning than splitting it into focused pieces. Conclusion: keep each pass small and distilled.
3. The synthesis. Combine them: fan out independent passes (fixes stochastic coverage) that are each small (fixes context fidelity), then have a stronger model fan them in (fixes "how do I trust a pile of overlapping answers?"). Use a cheap model for the many workers and an expensive model for the one judge, because that is where reasoning quality pays off.
4. Validation. This matches what Anthropic published independently — the orchestrator-workers and parallelization patterns, the Opus-lead/Sonnet-worker split, and the finding that separate context windows plus more sampled tokens drive most of the gain. (evidence)

Common mistakes

• Workers can see each other. Destroys independence; they converge on the first idea instead of sampling the distribution. Keep contexts isolated.
• Vague brief. Workers duplicate work or drift off-task. Give each an objective, an output format, and clear boundaries.
• Orchestrator re-reads all raw inputs. Defeats the purpose — it lands back in the context-rot zone. Workers must hand up short, distilled outputs.
• Same model for workers and judge. Wastes money (expensive workers) or under-powers the hardest step (cheap judge). Split the roles.
• Treating frequency as the only signal. A single worker with a verifiable fact can outrank a majority that is confidently wrong. The judge weighs evidence, not just counts.
• Using it for everything. It costs many× the tokens. Reserve it for hard, open-ended, or high-stakes problems.

Quick reference

References

• references/evidence.md — the published sources behind every claim above (Anthropic Building Effective Agents, Anthropic multi-agent research system, context-degradation research), with what each one supports.
• examples/example-invocation.md — copy-paste prompts to run the pattern.