The Fan-Out Tax

Step 1The known multiplier hides where the money actually goes.

The headline figure is well travelled: Anthropic's June 2025 write-up of its multi-agent research system put its token usage at roughly 15× a chat interaction, with an ordinary single agent around 4×. The same post reported the upside honestly — the multi-agent configuration beat a single Claude Opus 4 agent by 90.2% on their internal research eval, and token usage alone explained about 80% of performance variance — and named the condition: the economics only work where the task's value justifies the spend, and tasks whose agents must share context benefit least.

What the multiplier does not tell you is its shape. Teams read 15× as a constant and budget it as one: four branches instead of two, same multiplier, twice the speed. It is not a constant. It is the sum of three terms that scale differently in the number of branches, and only one of them is the work you wanted done. Separate them and the cap on useful fan-out falls out of the arithmetic rather than out of taste. This page assumes the supervisor shape from supervisor-worker; when and why to go multi-agent is the prior question of whether to fan out at all.

Step 2Term one: every branch re-pays a prefix the single loop had cached.

A sequential agent loop is cheap in a way that is easy to miss, because prompt caching does the work silently. The system prompt and tool definitions sit at the front of the context, unchanged across steps, so they are written to cache once and read back at a discount on every subsequent turn. Anthropic's published multipliers are a cache write at 1.25× the base input price for the default five-minute lifetime (2× for the one-hour option) and a cache read at 0.1× — a 90% discount on the bytes that do not change. Prompt caching has the mechanics and context-caching economics the break-even arithmetic.

A sub-agent's prefix is a different prefix. It has its own instructions, usually its own narrowed tool set, and its own slice of the task — which is exactly what what a subagent inherits is about, and it means the cache entry the parent built does not apply. Each branch opens with a miss and pays the write multiplier:

# the prefix term, in base-input-token equivalents
prefix        = 8000            # system + tool definitions, identical for every branch
steps_each    = 6
write_mult    = 1.25            # 5-minute cache write
read_mult     = 0.10            # cache read on every later step

# one sequential loop doing all 18 steps: one write, 17 reads
seq   = prefix * write_mult + prefix * read_mult * 17        # = 23,600

# three branches of 6 steps each: three writes, 5 reads apiece
three = 3 * (prefix * write_mult + prefix * read_mult * 5)   # = 42,000

# 1.8x on the prefix alone, before any of the real work is counted.
# six branches of 3 steps: 6 * (10,000 + 1,600) = 69,600, i.e. 2.9x

The term grows close to linearly in branches while the work per branch shrinks, so the prefix share of your bill rises every time you widen the fan. The practical consequence is a floor on branch length: a branch that runs three steps spends most of its budget introducing itself. If your prefix is large — a wide tool catalogue is the usual cause — fan-out is expensive before it has done anything, and trimming the per-branch tool set is a bigger lever than any model swap. KV-cache budgeting for agent loops is the single-loop version of the same accounting.

Step 3Term two: branch output is billed twice, and the second time it lands in the supervisor.

Whatever a branch concludes has to reach the supervisor, which means every token of every branch report is generated once at the output rate and then read again as input when the merge happens. That double billing is the same mechanism fewer tokens versus cheaper tokens identifies inside a single loop, but fan-out makes it structural: the merge's input is the sum of all branch outputs by construction, so the supervisor's own context grows with the branch count.

Three branches returning 2,000-token reports give the supervisor a 6,000-token read on top of its own prefix. Six branches give it 12,000, and now the merge step is the most expensive single call in the task — and the one most exposed to the effective-context problem, since a supervisor reasoning over twelve thousand tokens of six different narratives is in precisely the regime where recall degrades. That is the real ceiling on fan-out, and it is a context ceiling rather than a cost one.

So the merge budget, not the worker count, is what should cap the fan. Decide the largest merge input you are willing to reason over — call it 8,000 tokens — and derive the branch count from the report size you can actually enforce. If the arithmetic says four branches and you need twelve, the answer is not twelve branches: it is a reduce tree (merge in pairs, each merge producing one report of the same size) or a shared store the branches write to and the supervisor queries rather than ingests. Shared memory and the blackboard is that second option, and shaping tool results is how you make a report small enough to be worth returning at all.

Step 4Term three: branches cannot see each other, so overlapping discovery is paid per branch.

The third term is the one Anthropic's caveat points at. Independent branches rediscover the same things. Two sub-agents researching adjacent questions fetch the same three documents, read the same background, and each pay full price for it; the supervisor then reads both accounts of it in the merge. Nothing in the architecture deduplicates that, because the isolation that makes parallelism possible is the same isolation that prevents sharing.

This term is invisible in a token total and obvious in a trace. The measurement is cheap: count distinct retrieved documents or tool-call arguments across branches, and divide by the total. A ratio below about 0.7 — more than 30% of branch work is duplicated — means your decomposition is cutting across a shared dependency rather than along an independent seam, and re-cutting the task will beat any amount of per-branch optimisation. Multi-agent failure modes catalogues what else that bad seam causes, and the sub-agent patterns comparison covers which shape to reach for when the seam will not come clean.

A cheap partial fix: give the branches a shared read-through cache for retrieval — the same vector or document fetch, keyed and memoised at the tool layer, outside the model. It does not help the model-side duplication (each branch still reads the text into its own context) but it removes the retrieval and reranking spend, which on document-heavy work is frequently the larger line.

Step 5The latency side saturates, because the slowest of many is barely slower than the slowest of a few.

The benefit half of the trade is also misread. Wall-clock for a fan-out is the slowest branch plus the merge, not the mean branch — and that maximum grows as you add branches. For branch durations with a light tail, the expected maximum of k draws grows roughly like the mean plus a term in ln k, which has a specific and inconvenient consequence:

• 1 → 3 branches is the big win. Three sequential sub-tasks of duration T take 3T; in parallel they take about 1.2–1.4T plus the merge. Call it a 2× speed-up for a 1.8× prefix bill — a good trade on a task where latency has a price.
• 3 → 6 branches is not. The maximum moves from roughly 1.3T to about 1.5T, so you save nothing and may lose, while the prefix term goes from 1.8× to 2.9× and the merge input doubles.

Two real effects make this worse than the model suggests. Provider rate limits serialise part of your fan-out whether you planned it or not, so the sixth branch may not actually start when the first five do — the measured maximum then includes queueing. And branch duration in agent work is right-skewed, not light-tailed: one branch hitting a retry loop or a slow tool sets the whole task's clock. The honest summary is that fan-out buys the first factor of two in latency and very little after it. Price that factor properly before spending for it, because the dollar value of latency is not uniform — pricing latency shows the cost curve is nearly flat under a second, linear to about ten, and step-shaped after that, and measuring agent latency is how to get the distribution rather than a mean.

Step 6What to do: three conditions, one cap, three instruments.

Fan out when all three hold, and sequentially when any one fails:

• The branches are long. Enough steps that the per-branch prefix write amortises — as a rule of thumb, at least five or six tool-using steps each. Shorter than that and parallel tool calls inside one loop gives you most of the concurrency at none of the prefix cost, which is the option teams skip because it is less interesting.
• The branches are independent. Duplicate-work ratio above about 0.7 on a traced sample. Below it, re-cut the task.
• The branches compress. A report materially smaller than the work that produced it, with the size enforced in the sub-agent's instructions rather than hoped for.

Then cap the fan by merge budget rather than by worker count: pick the largest merge input you will reason over, divide by the enforced report size, and that is your branch count. Needing more means a reduce tree or a shared store, and both are cheaper than a supervisor drowning in six narratives. Keep a real termination condition on every branch, because a step cap turns a slow branch into a silently truncated one and the merge cannot tell the difference.

Three instruments tell you whether a given fan-out is earning its keep, and none of them is total token count:

• Cache-hit rate per branch. If branches are hitting cache less than the single loop did, you have found the prefix term in your own traces. Falling hit rate as fan-out widens is the signal that the fan is too wide.
• Merge input as a share of total input. Rising means you are paying to read your own output. Past roughly a third, restructure before optimising.
• Seconds saved per extra dollar. The actual exchange rate of the trade, measured against a sequential baseline you still run on a sample. It is the only number that answers "should the fourth branch exist", and it is almost always worse than the third branch's.