The price cliff above the context threshold.
Long-context pricing is a step function, not a slope: cross a per-request token threshold and the higher rate applies to every token in that request, so a run that grows 2.5% can cost 108% more. Worse, the cached prefix you added to save money counts toward the threshold and is itself billed at the doubled cache rate — so the optimisation is what pushes you over. The lever is a cap on request size, not a cheaper model.
Step 1The meter is a step, and the step applies to the whole request.
Most cost reasoning about agents assumes the token meter is linear: twice the context, twice the input bill. For the long-context models that agents actually want, it is not. Google's published pricing lists two separate rate columns for Gemini 3.1 Pro and 2.5 Pro — one for prompts at or under 200,000 tokens, one for prompts above it — and as of November 2026 the 3.1 Pro preview rates are $2.00 per million input tokens under the line and $4.00 above, with output moving from $12 to $18. Flash and Flash-Lite tiers are flat regardless of prompt length.
The part that changes your arithmetic is not the doubling. It is that the threshold selects a rate for the entire request rather than pricing only the overage: a 201,000-token prompt does not bill 200,000 tokens cheaply and 1,000 expensively. Every token in it moves to the upper rate. That is what makes this a cliff rather than a surcharge, and it is why a cost model built from an average context length will be wrong in both directions — too high for the runs that stay under, far too low for the ones that tip over.
The tier columns come from the vendor's own pricing table and are the solid part. That the upper rate applies to the whole request rather than the overage is stated consistently by third-party analyses but we could not trace it to a sentence in the official documentation, and the same is true of whether cached tokens count toward the threshold — STEP 3 shows how to settle both with two API calls against your own account. Rates move; re-read the table before you commit a forecast.
Step 2Work the example, because the shape is counter-intuitive.
Take a coding or research agent with a large pinned prefix — repository map, instructions, retrieved documents — cached at 180,000 tokens, plus 25,000 tokens of fresh tool results and conversation, producing 2,000 tokens out. That is a 205,000-token request: 2.5% over the line.
at 180k cached + 20k fresh = 200k (under the line)
cached 180,000 @ $0.20/M = $0.0360
fresh 20,000 @ $2.00/M = $0.0400
output 2,000 @ $12.00/M = $0.0240
-------
$0.1000
at 180k cached + 25k fresh = 205k (over the line)
cached 180,000 @ $0.40/M = $0.0720
fresh 25,000 @ $4.00/M = $0.1000
output 2,000 @ $18.00/M = $0.0360
-------
$0.2080
+2.5% tokens -> +108% cost on the call
Run that through a fleet and the consequence is a bill with no visible cause. At 20,000 calls a day the two profiles are roughly $2,000 and $4,160 a day — a $65,000-a-month difference produced by five thousand extra tokens of tool output that nobody shipped a change for. And note which direction the cheap fix runs: trimming 6,000 tokens of tool results, about 3% of the request, cuts the call in half. That is a return no model swap offers, which is the opposite of the usual advice in fewer tokens versus cheaper tokens — near a cliff, fewer tokens wins by a margin that has nothing to do with the per-token rate.
Step 3The cache is the usual culprit, and it is the thing you added on purpose.
This is the part that catches careful teams. A prompt cache cuts the rate on a repeated prefix by an order of magnitude — Gemini's cached-input rate is a tenth of standard in both tiers — so the rational move is a large stable prefix, and prompt caching is near-universal advice for agent loops. But the cached tokens are still in the request. They almost certainly count toward the threshold, and the cache rate itself has two tiers: $0.20 per million under the line and $0.40 above.
So a 180,000-token cached prefix is not a cost optimisation that happens to be large. It is a permanent position 90% of the way up the cliff, where every turn's ordinary growth — one more tool result, one more retrieved document, the reasoning carried from the previous step — crosses the line and doubles the rate on all 180,000 cached tokens too. The discount you bought is still there; it is just being applied to a doubled base.
Two consequences for how you size a prefix. Treat the threshold as the budget, not the window: if your model's cliff is at 200,000 tokens, a prefix over about 120,000 leaves no room for a run to breathe and should be split or trimmed rather than cached whole. And measure, do not assume — send one request at 199,000 tokens and one at 201,000 with the same cached prefix and compare the billed amounts. Two calls settle for your account and model what no blog post can, including the question of whether cached tokens count. The storage meter is separate and time-based, around $4.50 per million tokens per hour on Gemini's table; context-caching economics covers that side.
Step 4Agents cross thresholds by accident, which is why this is an operations problem.
A chat product's prompt length is roughly a function of the user. An agent's is a function of how the run went, and nothing in the loop knows there is a price edge in it:
- Tool results are unbounded by default. One verbose API response, one large file read, one page of search results, and the request jumps by tens of thousands of tokens — the problem large results and pagination exists to bound.
- Reasoning tokens accumulate. When reasoning is carried across tool calls it enters the next request as input, so the request grows even on turns where no new information arrived; see reasoning token accounting.
- Retry amplification multiplies the expensive calls. A failed call at the upper tier is retried at the upper tier, because the retry carries the same oversized context — retry amplification, now with a rate multiplier on it.
- Compaction fires late. If your compaction trigger is a fraction of the window — say 80% of 1M — it will not fire until long past a threshold at 200K. The trigger has to be set against the price edge, not the capacity limit.
Which makes the fix a configuration change rather than an architecture change. Put a hard cap on request size below the threshold, and make the thing that enforces it the same mechanism that already manages the window: context compaction triggered at the cliff, context budgeting for the allocation, and a truncation rule on tool results so a single response cannot tip a run on its own.
Step 5Vendor behaviour differs and changes, so treat the cliff as a per-model fact.
There is no cross-vendor rule to learn here, which is itself the useful finding: the threshold is a per-model pricing decision that moves between generations, so it belongs in your model metadata next to the context window and the rate.
As of November 2026, Gemini's Pro models carry the two-tier structure at 200,000 tokens while the Flash tiers are flat. Anthropic's position is the one the secondary sources most disagree about — some report that an earlier 2×-above-200K surcharge has been dropped for recent Claude generations and that the full million-token window bills at standard rates, with a lower threshold reported for the smallest model; we could not resolve that from the vendor's own table and you should not plan against any version of it without checking. OpenAI's handling is likewise reported both ways for different model generations.
Operationally, the way to stop caring which is true today: record the threshold and both rate pairs as fields on each model you call, assert them in the same test that checks your pricing table, and alert when a request exceeds the recorded threshold. That is the same discipline as tokenizer changes and your cost baseline — a vendor-side change to how your tokens are counted or priced should fail a test, not surface as a bad month. And when a model is retired, the replacement's cliff may sit elsewhere, which belongs in the checklist in model deprecation and migration.
Step 6Make the cliff visible in the numbers you already watch.
Average cost per run hides this completely, because the distribution is bimodal: most runs sit under the line and a minority sit above it at roughly double, and the mean lands somewhere neither population occupies. Three instrumentation changes make it legible, and all three are cheap:
- Record request input tokens as a distribution, not a mean, and plot the threshold on it. The number you want is the share of calls above the line; it is actionable in a way an average is not, and it moves when someone widens a tool result. Where those spans roll up is cost attribution and budgets.
- Tag each call with the tier it billed at, so cost per task can be split by tier. A rising bill with a flat tier mix is traffic growth; a flat bill with a rising tier mix is a prompt change about to become expensive.
- Alert on the share, not the spend. Spend confounds volume with rate. Tier share isolates the thing a prompt or tool change actually moves, which is what makes it a usable signal in forecasting agent spend.
Do this in one sitting. Look up the threshold and both rate pairs for every model in production and write them into your model metadata — for several models the answer will be "no threshold", which is worth knowing explicitly. Then plot last week's request input-token distribution against each threshold: if no calls are within 20% of a line, stop here, you do not have this problem. If a tail is crossing, set compaction to fire at about 90% of the threshold rather than at a fraction of the window, cap any single tool result at a few percent of it, and re-run. Only then consider a model with a flat rate, because moving model to dodge a cliff you could have capped is the expensive version of this fix.