Fewer Tokens vs Cheaper Tokens

Why the two discounts are not interchangeable.

Vendors advertise two kinds of saving and procurement treats them as fungible. They act on different parts of the bill.

• A price cut is multiplicative on everything. Every token on the invoice gets cheaper: the system prompt, the tool schemas, the retrieved documents, the tool results, the re-sent transcript and the model's own output. Nothing is exempt.
• A brevity win is multiplicative on one term. Fewer generated tokens shrinks the model's output and the part of the transcript that output becomes. It does nothing to the stable prefix, which is re-sent on every step regardless, and nothing to tool results, which are the environment's output rather than the model's.

In an agent loop those untouchable terms are large, because the loop re-sends everything on every step — the quadratic arithmetic in agent cost control. A twenty-step task with a 4,000-token prefix pays for that prefix twenty times, and a 600-token tool result arriving at step three is re-read at steps four through twenty. Brevity cannot reach any of it.

The marginal cost of a generated token, in closed form.

Write a token at step i of an N-step task. You pay the output rate once, and then the input rate on every one of the N − i steps that re-send the transcript containing it. So:

cost(one token written at step i) = p_out + (N - i) * p_in_effective

# N = 20, p_in = $0.60/M, p_out = $2.40/M, no caching
# average over the trajectory: p_out + p_in * (N-1)/2
#                            = 2.40 + 0.60 * 9.5
#                            = $8.10 per million  ->  3.4x the output rate

Three things follow immediately, and all three are operationally useful.

• Verbosity is roughly three to four times more expensive than the output price suggests on a medium-length trajectory, which is why reasoning-heavy models surprise people on the invoice rather than on the rate card — the accounting problem in accounting for reasoning tokens.
• Early tokens cost more than late ones. A token written at step 2 of 20 is re-read eighteen times; one written at step 19 is re-read once. A verbose planning phase is the most expensive verbosity you can buy, and a verbose final answer is nearly free.
• The multiplier is linear in trajectory length. Halving the number of steps halves the re-read penalty on every token, which is why capability that finishes sooner is a cost lever and not a luxury.

Pass-through: what a 3× brevity win is actually worth.

Model one task end to end. Twenty steps, a 4,000-token stable prefix, 600-token tool results, input at $0.60 and output at $2.40 per million, no caching. One model writes 1,200 tokens a step; the other writes 400 — a clean 3× on generated tokens.

input tokens  = N*prefix + (write + tool_result) * N*(N-1)/2
output tokens = N*write

                           input     output      bill   vs base
verbose, 1200/step       422,000     24,000    $0.311     1.00x
brief,    400/step       270,000      8,000    $0.181     1.72x
verbose at 1/3 price     422,000     24,000    $0.104     3.00x

A 3× cut in what the model writes bought a 1.72× cut in the bill. A 3× price cut bought 3.00× — and beat the brief model outright, $0.104 against $0.181. The brief model only wins here if it is also cheaper, or if it finishes in fewer steps.

One number explains the whole table. Call α the share of the bill that the model's own writing controls — its output tokens plus the re-reads of those tokens — as against the prefix and the tool results, which it does not control:

alpha = (p_in * N*(N-1)/2 + p_out * N) * write
        / total_bill

saving from writing b times fewer tokens  =  1 / ((1 - alpha) + alpha/b)
saving from paying q times less per token =  q          (exactly)

# worked above: alpha = 0.63, b = 3  ->  1.72x

So brevity is capped by α and a price cut is not capped at all. The break-even is blunt once you write it down: a 3× brevity win beats a mere 2× price cut only when α exceeds about 0.75 — when three quarters of your bill is the model's own prose and its echoes. Uncached, with a real prefix and real tool results, almost no agent is there.

Prompt caching raises α, so it makes brevity matter more — not less.

The intuition most teams have is that caching makes verbosity harmless, since the re-reads become cheap. The arithmetic says the opposite, and the reason is that caching discounts precisely the part of the bill brevity could never touch.

Take the same task with 90% of input served from cache at a tenth of the input rate, so the effective input price is $0.114 per million:

                           input     output      bill   vs base
verbose, 1200/step       422,000     24,000    $0.106     1.00x
brief,    400/step       270,000      8,000    $0.050     2.11x

alpha rises from 0.63 to 0.79   ->  brevity's payout rises from 1.72x to 2.11x
output goes from 19% of the bill to 54% of it

The stable prefix was a floor under both models' bills; caching removes most of the floor, so what remains is dominated by the thing the two models differ on. Above the α ≈ 0.75 threshold, the 3× brevity win now does beat a 2× price cut. Three consequences for how you sequence cost work:

• Turn caching on before you shop for a terse model. Caching is the cheapest lever available and it changes the answer to the model question — see prompt caching. Doing it the other way round means choosing a model against the wrong α.
• Caching and brevity are complements, not substitutes. They act on different terms, and each one raises the return on the other. Most cost write-ups treat them as alternatives.
• A cache miss rate is an α mover, so watch it. If a change to the prefix invalidates the cache, α falls, and the terse model you paid a premium for stops earning it. Put the hit rate on the same dashboard as the spend, per cost attribution.

What brevity buys that money cannot.

The bill is the weakest case for a terse model, and if it were the only case the conclusion would simply be "buy cheaper tokens". Three other effects do not appear on an invoice at all, and two of them are the reasons teams actually switch.

• Latency, which is nearly linear in tokens written. Output decodes serially, so three times fewer tokens per step is close to three times less time per step. A price cut buys none of this. On a twenty-step task it is the difference between a tool someone waits for and one they abandon — priced in pricing latency.
• Context headroom, which is a correctness property. Every token written occupies the window for the rest of the run, so a verbose model reaches compaction sooner, and compaction is where agents lose the thread — context compaction. Effective recall also degrades well before the advertised limit, so a shorter transcript is more accurate as well as cheaper: effective versus advertised context.
• Step budget, because tokens and steps are rationed together. A fixed token ceiling converts directly into a number of steps the agent gets, so a terse model is allowed more attempts inside the same cap — which is the evaluation consequence in budget-matched evaluation.

And there is a lever that dominates both discounts and is usually available: write less context in the first place. Returning a tool result by reference rather than by value, truncating at the tool boundary and summarising old turns all reduce the terms brevity cannot reach and caching only discounts — the whole subject of context engineering and context budgeting.

The decision rule, and the five numbers it needs.

You cannot convert any vendor efficiency claim into your own bill without five figures from your own traces, and all five are already in your telemetry if you record per-run token accounting at all.

• Median steps per completed task. This sets the re-read multiplier, and it is the figure most teams have never looked at.
• Prefix size — system prompt plus tool schemas plus anything else constant.
• Median tool-result size, which is the other term brevity cannot touch.
• Median tokens written per step.
• Cache hit rate on input.

Put them into the α expression in STEP 3 and the rule falls out. Below α ≈ 0.6, buy price: route the high-volume steps to a cheaper model and keep the frontier model for the reasoning, per model routing and small and local models. Above α ≈ 0.8, buy brevity — and consider paying a premium for it, because at that point it is also buying you latency and context headroom. In between, the two levers are within noise of each other on cost, so decide on latency instead.

Two cautions on the measurement. First, α is per workload, not per company: a research agent with a long trajectory and a classification agent with a one-step call sit at opposite ends, so compute it per surface and expect different answers — the discipline in unit economics. Second, none of this survives a change in success rate. A terse model that fails more often is paying full price for failed attempts and then paying again for the retry, which swamps every factor in this essay; measure cost per successful task or you will optimise your way into a cheaper system that delivers less.

Related: price deflation and cost per task for why the rate card keeps falling while your bill does not, forecasting agent spend for turning α into a budget, and adaptive thinking and effort budgets for buying brevity from a model you already have.