Vendors moved from token-budget dials to adaptive effort levels in 2026 — Anthropic deprecated budget_tokens — and the three vendor idioms are close enough to compare but different enough to trip you up.
Every "budget your thinking tokens" example from 2024-2025 is stale. Anthropic deprecated budget_tokens on Opus 4.6 / Sonnet 4.6 and removed it on 4.7+; the replacement is thinking.type: adaptive with effort: low|medium|high. Gemini uses thinking_level. OpenAI uses reasoning_effort. Same idea, three vocabularies, one gotcha — the model can override your budget when its own gradient says it needs more. This essay is the API surface, the override behavior, and the cost math per vendor.
Why token budgets got replaced.
The first generation of reasoning APIs — Claude 3.7 extended thinking, o1 in its initial GA shape, Gemini 2.5 Flash Thinking — all shared one control surface: a numeric ceiling on the private thinking tokens the model was allowed to spend before it had to emit an answer. The names varied (budget_tokens, max_reasoning_tokens, thinking_budget) and the defaults varied, but the shape was the same. You picked a number, the model spent up to that number, and if it hit the ceiling it stopped thinking and produced whatever answer the truncated trace supported. This was legible and it composed well with existing token-budgeting code paths, and it was wrong in the same way a fixed test-time compute budget was wrong in the previous essay of this group — a uniform ceiling wastes budget on the easy majority of queries and starves the hard tail that would actually earn the spend.
The specific failure mode is easy to reproduce. Set budget_tokens=8000 on a mixed workload and log the actual thinking tokens consumed per query. On lookup-shaped questions the model spends 200 tokens and returns; on straightforward derivations it spends 1500; on the one hard problem in twenty it wants 12000 tokens and gets cut off at 8000, delivering a partial answer that's often worse than a single-pass no-thinking response would have been. Raise the ceiling to 16000 and you pay 2x on every query that never needed it. There is no single number that is right for the mixed workload — the compute-optimal test-time policy from the inference-time scaling essay says the budget should be a function of estimated per-query difficulty, and a numeric ceiling can't encode that function.
The vendor move in 2026 is to shift the difficulty estimator inside the model. Instead of exposing a token dial, expose a coarse intent — how hard should the model try — and let the model translate that intent into an actual token spend per query, calibrated to the query it's about to answer. Anthropic named the shape first (effort: low|medium|high), Gemini and OpenAI followed with their own words for the same idea. The resulting API is less controllable in one dimension (you can no longer force exactly N tokens) and much more efficient in another (the low-effort dial spends a couple hundred tokens on easy queries and the high-effort dial spends five figures on hard ones, from the same setting). The when reasoning helps essay describes the same shift at the policy layer — "set the thinking budget to X for query class Y" was already the right advice; the API is now catching up to make X mean an effort level instead of a token count.
The Anthropic effort API.
Anthropic's shape on Claude Opus 4.7 and Sonnet 5 is a thinking object on the request, with type: "adaptive" and an effort enum. The older type: "enabled" shape with budget_tokens is still accepted on 4.6-class models with a deprecation warning; on 4.7+ it returns a 400. The migration is one field rename plus dropping the number, and every SDK release from late-May 2026 onward defaults new code to the adaptive shape.
POST /v1/messages HTTP/1.1
Host: api.anthropic.com
anthropic-version: 2026-05-01
content-type: application/json
{
"model": "claude-opus-4-7",
"max_tokens": 4096,
"thinking": {
"type": "adaptive",
"effort": "high"
},
"messages": [
{"role": "user", "content": "Prove that sqrt(2) is irrational."}
]
}
The three effort levels are calibrated targets, not fixed budgets. low aims at a few hundred thinking tokens on easy queries and up to a couple of thousand on harder ones; medium is the default for reasoning-mode requests and roughly matches Claude 3.7's midrange extended-thinking spend; high can consume tens of thousands of thinking tokens on a hard problem and will happily spend zero on a lookup. The model chooses per-query. The effort value shifts the entire distribution up or down, so the same query will spend more thinking tokens at high than at low, but the ratio between an easy and a hard query at any given effort level is set by the model rather than by you.
Two migration notes matter more than the field rename. First, the response now carries an actual_effort field alongside the streaming thinking blocks that reports what the model actually chose to spend — the value is one of the same three enums, and it can be lower than what you requested (the model decided the query didn't earn the escalation) but not higher. Log this field on every request; the observability story for adaptive thinking is impossible without it. Second, cache-control on system prompts and tools is unchanged, but the thinking block itself is never cached — a query that spends 12000 thinking tokens pays for those tokens every time it runs, even if 90% of the prompt was cache-hit. The context caching economics essay walks through how this interacts with the write premium; the short version is that adaptive thinking and caching are additive levers, not multiplicative ones.
Gemini thinking_level.
Gemini 3.1 Pro and 3.5 Pro use a thinking_level field on the generation config, with the same three values in a slightly different vocabulary: off, low, medium, high. The off value is the important addition — Gemini's default on Flash-class models is a low level of thinking rather than none, and if you want strict zero-thinking behavior (parity with a non-reasoning model) you have to name it explicitly. The Pro-class models default to medium. The older thinking_budget integer field is still on the API and still works, but the docs mark it as legacy and future model versions drop it.
POST /v1/models/gemini-3.5-pro:generateContent HTTP/1.1
Host: generativelanguage.googleapis.com
content-type: application/json
{
"contents": [
{"role": "user", "parts": [{"text": "Prove that sqrt(2) is irrational."}]}
],
"generation_config": {
"max_output_tokens": 4096,
"thinking_level": "high"
}
}
Gemini's semantics diverge from Anthropic's in two ways worth pinning down. First, the response separates thinking tokens into thoughts_token_count on the usage block, distinct from output_token_count, and both bill at the standard output-token rate — the same as Anthropic and OpenAI. Second, Gemini exposes a thinking_summary flag that returns a compressed, human-readable summary of the reasoning trace in the response rather than the raw trace itself; this is useful for UI surfacing and for judge-based evaluation but is not a replacement for the raw trace when you need to debug a specific failure. The reasoning models concept covers the raw-vs-summary trade-off in more depth.
The one gotcha specific to Gemini is that the thinking level interacts with tool calling in a way the other vendors' don't. At high, Gemini will happily emit multiple rounds of thinking interleaved with tool calls on the same request — think-call-think-call-answer — and the total thinking budget across all rounds can be larger than any single-round budget would be. This is a feature for agent-shaped workloads (the model gets to reason about tool results before the next call) and a cost surprise for teams that budgeted assuming one thinking block per request. If you're wiring Gemini into an agent loop, model your token cost as "N tool rounds × per-round thinking spend" rather than a fixed per-request number.
OpenAI reasoning_effort.
The OpenAI shape on o3 and o4-mini uses a reasoning object on the request with an effort field, and the three values are low, medium, high — the same enum names Anthropic uses. The similarity is deliberate; the shift from the older reasoning_effort top-level field to a nested reasoning.effort was published as an "align with vendor conventions" note in the SDK changelog. The older flat field still works on o1 and o3-mini but is not accepted on o4-mini or GPT-5.5. There is no off value — reasoning models always reason to some degree; if you need zero reasoning you use a non-reasoning model.
POST /v1/responses HTTP/1.1
Host: api.openai.com
content-type: application/json
{
"model": "o4-mini",
"max_output_tokens": 4096,
"reasoning": {
"effort": "high"
},
"input": [
{"role": "user", "content": "Prove that sqrt(2) is irrational."}
]
}
OpenAI's semantics diverge from Anthropic's in one important place: the reasoning trace is not returned to the client by default. The response contains a reasoning block with a summary (short natural-language description of what the model reasoned about) and a encrypted_content field (opaque to you, replayable on subsequent requests to preserve reasoning continuity across turns), but not the raw thinking tokens themselves. You are billed for those tokens — the usage block reports reasoning_tokens — but you cannot inspect them. For debugging, the summary is what you have. For evaluation, this is a real limitation compared to Claude's raw traces and Gemini's opt-in summaries; the chain-of-thought faithfulness discussion is harder to apply when you can't read the chain.
The encrypted_content mechanism is worth understanding because it changes how multi-turn reasoning conversations bill. On a follow-up turn, you send back the previous turn's encrypted_content and OpenAI decrypts and prepends the earlier reasoning to the new turn's context. This preserves reasoning continuity (the model doesn't re-derive what it already worked out) at the cost of the earlier reasoning tokens being billed again as input tokens on the follow-up. The trade is usually worth it for agent-shaped workloads where the model builds on prior reasoning; it's a waste for stateless one-shot queries. Watch this on cost reports — a chatty multi-turn conversation with high-effort reasoning on every turn can produce a bill that is dominated by re-sent encrypted reasoning rather than by fresh generation.
When the model overrides your budget.
The single most-missed detail across all three vendors is that the effort level you request is a hint, not a hard cap. The model can decide the query is genuinely hard and spend more thinking tokens than the level would typically produce; conversely (more commonly), the model can decide the query is easy and spend far less than the level suggests. Anthropic documents this explicitly with the actual_effort response field mentioned in STEP 2. Gemini documents it implicitly through the observation that thoughts_token_count distributions at any given level have a long tail. OpenAI documents it by acknowledging that reasoning_tokens at high effort can exceed the fixed thinking budgets that the older API let you set.
The override behavior is rare in the low-and-medium range and real at high. On low-effort settings the actual thinking spend hugs the requested level tightly; the model almost never decides an easy query needs more thought than "low" indicates. On high-effort settings the model is willing to escalate — an especially gnarly derivation or a query that turns out to require multi-step planning can spend two or three times what a typical "high" query does. The vendors' framing is that this is the whole point of adaptive thinking: fixed budgets underspent on the hard tail, and the adaptive dial recovers that spend without asking you to guess when to grant it.
# Actual thinking tokens on a mixed workload of 1000 queries # claude-opus-4-7, thinking.effort = "medium" percentile thinking_tokens actual_effort p10 180 low p50 1450 medium p90 4200 medium p95 7800 medium p99 18400 high <- model escalated p99.9 42100 high <- model escalated # Total spend: 1.9M thinking tokens across 1000 queries # Naive equivalent (budget_tokens=8000): 8.0M tokens billed # Adaptive savings: ~76% on this workload shape
Two consequences for cost accounting follow. First, thinking-token spend is not predictable from the request; you have to measure it. Build a per-workload distribution of actual_effort and thinking-token counts before you commit to an effort level in production, because the model's behavior on your queries is what determines the bill, not the level string. Second, high-effort escalation is where the p99 cost tail sits — a workload that mostly runs at medium spend but occasionally escalates at p99 has a bill dominated by that tail, and rate-limiting or capping strategies (fall back to medium if the last query cost more than N tokens) are the operational mitigations that keep the bill within a budget you can commit to. The observability question is not "what did I request" but "what did the model choose to spend"; instrument both.
Cross-vendor comparison table.
Read the three shapes together and the seams are obvious. The names differ, the enums differ slightly, the default behavior differs, and the trace visibility differs the most. What follows is the field-by-field mapping a portable client has to encode; every multi-vendor stack ends up with a small adapter layer that translates a single internal "effort" concept into the three shapes below.
| Field | Anthropic | Gemini | OpenAI |
|--------------------|-------------------------------|-------------------------------|-------------------------------|
| Container | thinking: {} | generation_config.thinking_* | reasoning: {} |
| Enable adaptive | type: "adaptive" | (implicit; set thinking_level)| (implicit; set effort) |
| Effort field | effort | thinking_level | effort |
| Values | low | medium | high | off | low | medium | high | low | medium | high |
| Zero-thinking | (use non-reasoning model) | thinking_level: "off" | (use non-reasoning model) |
| Deprecated | budget_tokens (removed 4.7+) | thinking_budget (legacy) | reasoning_effort top-level |
| Actual spend field | usage.thinking_tokens | usage.thoughts_token_count | usage.reasoning_tokens |
| Actual effort | response.actual_effort | (inferred from token count) | (inferred from token count) |
| Trace visibility | raw thinking blocks | opt-in summary | summary + encrypted opaque |
| Multi-turn replay | resend full trace or drop | resend summary or drop | resend encrypted_content |
| Interleaved tools | one thinking block per turn | multiple rounds per request | one reasoning block per turn |
| Billing | output-token rate | output-token rate | output-token rate |
Three cross-cutting notes. The billing rate is uniform: thinking tokens bill at the same rate as output tokens on all three vendors, and none of the three currently discounts them (batch discounts still apply, cache discounts do not apply to the thinking block itself). The multi-turn story is where the three diverge most operationally — Anthropic's replay-or-drop is simplest to implement, Gemini's summary is cheapest on the wire, OpenAI's encrypted-content is the most seamless for continuous reasoning but the least transparent. The interleaved-tools row is the one to internalize: Gemini's multi-round-per-request shape means a single API call can produce several thinking blocks separated by tool calls, and your token budgeting has to reflect that.
Cost implications.
Adaptive thinking changes the cost model in three ways that the fixed-budget generation did not. First, the per-query cost is now stochastic — the model decides how much to spend, so your unit economics have to be modeled as a distribution rather than a point estimate. A workload with a mean of 1500 thinking tokens per query might have a p99 of 20000, and the bill is dominated by which side of that distribution your traffic sits on. Second, the effort dial is where you should be spending your optimization time — moving from high to medium on a workload where the model was rarely escalating anyway drops the mean spend by 50-70% with a barely-measurable accuracy delta, which is a bigger lever than most caching or batch-optimization work. Third, thinking tokens do not compose with prompt caching — the cached prompt cuts the input-token bill, but the thinking tokens are always freshly generated and pay full output-rate on every query.
A concrete pattern that ships in production: run the workload at medium as the default; for the small subset of queries that a difficulty estimator flags as "genuinely hard" (heuristics: long chain of dependencies, tool-heavy, or classified as such by a lightweight classifier), promote to high; for the queries the estimator flags as trivial (short lookup, single-fact retrieval), demote to low. This is the same difficulty-adaptive policy the inference-time scaling essay described for external compute, applied to the internal reasoning dial. On mixed workloads that split roughly 70% easy / 25% medium / 5% hard, this policy typically produces a 40-50% cost reduction versus a uniform medium setting with an accuracy delta below 1 point on standard benchmarks — the numbers are workload-dependent, but the shape of the win is consistent across the three vendors.
The observability discipline that makes this tractable is small. Log the requested effort, the actual thinking-token count, and (where available) the reported actual-effort field on every request. Bucket the results by workload class and by input-length band. Chart p50 / p95 / p99 thinking tokens per class, and you will see the escalation events where the model chose to spend more; those are the queries to inspect for whether the spend was earned or whether a lower effort level would have produced the same answer. The three vendors' APIs make this instrumentation easy — a couple of extra fields in the request log — and the payoff is that you know, per workload, whether adaptive thinking is buying you accuracy or just billing you more. The deprecation of budget_tokens forced this discipline into everyone's roadmap; the shape the three vendors converged on rewards teams that treat the effort level as a per-query decision rather than a global setting.