Thinking tokens are billed at output rate — an adaptive `effort: high` on a task that didn't need it 3-5× the bill without changing the answer, and the decision rule is empirical, not principled.
By 2026 every major vendor's reasoning API takes an effort (Claude) / thinking_level (Gemini) / reasoning_effort (OpenAI) dial. The default answer is "just leave it on medium." That default is expensive — thinking tokens are billed at output rate, and on latency-sensitive paths medium can 2× your latency budget with no measurable quality gain. This chapter is the decision rule that survives contact with a real cost sheet: which task classes actually benefit from thinking, which ones burn money on it, and how to measure the delta on your specific traffic before you commit to a setting. By the end you'll have a per-task-class thinking policy and know why "default medium" is the most expensive mistake.
The three vendor dials.
Every major reasoning API in 2026 exposes the same abstract knob under a different name. Claude calls it effort, with levels low, medium, and high. Gemini calls it thinking_level, with the same three tiers plus a none that suppresses reasoning entirely. OpenAI calls it reasoning_effort, with minimal, low, medium, and high. The mental model behind all three is identical: a hint the runtime uses to allocate a variable number of hidden reasoning tokens before it emits the visible answer. The adaptive thinking and effort budgets deep-dive walks the internals; for this chapter you only need to know that the dial is a hint, not a contract, and the model reserves the right to spend more or fewer tokens than the tier suggests when it disagrees with your framing of the task.
The older interface — Anthropic's budget_tokens — is deprecated as of the 2026 spec because it forced authors to reason about token accounting rather than the outcome they cared about. The tier abstraction is a better shape, but it hides an operational fact worth naming: thinking tokens are billed at output rate. On Claude that means every hidden reasoning token costs the same as an emitted output token, and effort: high on a task the model handles cleanly at effort: low is a straight 3-5× increase in the output-token line of your bill for no answer-quality change. That fact — combined with the reasoning-vs-non-reasoning-models mental model of what the tokens are actually doing — is the whole basis for the decision rule in the sections that follow.
Task classes that benefit from thinking.
Thinking pays off when the task has a verifiable chain of intermediate steps and the model routinely gets the last step wrong without them. The when reasoning helps deep-dive walks the taxonomy in full; the short list of classes where medium-or-higher effort earns its bill is compact and consistent across vendors.
Multi-step logic and formal reasoning — proofs, SAT-shaped constraint problems, multi-hop deductions — are the paradigm case. The scratch space is the answer. Turning effort off on these classes drops accuracy sharply and predictably, and the extra latency is buying a real quality win. Math with more than one operation belongs here too: single-step arithmetic doesn't benefit, but any problem where a wrong intermediate step propagates to the final answer does. If you can grade the intermediate steps and the model gets them measurably better with effort on, keep it on.
Code generation with tests is the next-clearest win. When the task involves producing a function whose behavior is checked by a runnable test suite, medium or high effort lets the model iterate against its mental model of the tests before emitting the final draft. On agentic coding harnesses — the kind covered by the tool-use chapter — this often translates into fewer edit-run-fix cycles overall, which reduces the total token spend across the loop even though each step costs more. The escalation is worth watching: if you find the model is spending high-effort thinking on a step where a simple lint fix would suffice, dial down.
Agentic tool-loops with credit assignment are the third class. When the model has to decide which of several tools to call, then reason about the result of the last call before choosing the next, thinking tokens help it maintain the chain. This is why long-horizon research agents and browser agents (see r2 Computer Use in Production) commonly ship with effort on: the incremental cost is small compared to the cost of a wrong tool choice five hops in.
Task classes where it burns money.
The larger surface — the one that catches teams by surprise on their first cost review — is the set of classes where thinking helps roughly zero while costing three to five times as much. These are the money-burning defaults, and every one of them is a place where "just leave it on medium" turns out to be the most expensive setting available.
Extractive question-answering is the top offender. When the answer is a span of the prompt — "what's the customer's account number in this email" — the model finds it in one attention pass and any additional thinking is spent second-guessing a correct first answer. On production traffic dominated by extraction, moving from medium to off typically drops per-call cost by 3× with no measurable regression on a golden set.
Structured output from a fixed schema is the second. When the task is "convert this free-text description into the following JSON shape," the schema is the reasoning, and the model already applies it deterministically. Thinking tokens can actively hurt here because they occasionally encourage the model to reshape the schema on the fly. If your JSON validator is bouncing more responses at medium than at off, the extra reasoning is not being spent on the thing you asked for.
Most classification tasks belong to the same family. Sentiment, intent, category — anything where the label set is small and the input is short. The 2026 benchmark data on classification effort curves is unambiguous: quality plateaus below medium for the vast majority of production classifiers, and the cost curve keeps rising. Summarization, translation of short passages, and simple rewrites round out the list. None of these tasks have the "verifiable intermediate step" property that makes thinking pay.
How to measure the delta on your traffic.
The taxonomy above is a strong prior, but every production traffic shape has its own idiosyncrasies, and the right way to commit to a setting is to measure on your own data before you flip the dial. The measurement is a small A/B: replay a sample of production inputs at two effort settings, score the outputs against a fixed judge or a golden set, and compare quality delta against cost delta.
# A/B replay — 500 sampled prompts, one task class at a time config: class=customer-email-classification, n=500 setting A: effort=off -> cost=$0.42 latency_p50=380ms quality=0.94 setting B: effort=medium -> cost=$1.61 latency_p50=920ms quality=0.94 delta: cost x3.8 latency x2.4 quality +0.00 decision: effort=off wins (no quality gain, 3.8x cost, 2.4x latency) config: class=multi-hop-reasoning-audit, n=500 setting A: effort=off -> cost=$0.61 latency_p50=440ms quality=0.71 setting B: effort=medium -> cost=$2.30 latency_p50=1180ms quality=0.89 delta: cost x3.8 latency x2.7 quality +0.18 decision: effort=medium wins (quality gain justifies cost)
Two rules of thumb make the read easier. If the quality delta is below 5% and the cost delta is above 2×, off wins by default — reserve effort for the classes where the delta clears the noise floor. If the quality delta is above 15%, effort is almost certainly worth it and the question becomes medium vs high, which is a second A/B with the same shape. Anything in between is a business call: how much is a percentage point of quality worth on this class? For a fraud-detection classifier the answer might be "any cost is fine." For a bulk email tagger the answer is "not much." Push the number to your cost-and-latency budget from s2 Cost & Latency and let that decide.
A per-task-class thinking policy.
The output of the exercise above is a policy — a table your code consults before it sets the dial. Keep it explicit, keep it versioned with your prompts, and revisit it whenever a vendor releases a new model tier or you notice the underlying traffic distribution has shifted. A worked example, expressed as configuration.
# thinking_policy.yaml — per-task-class effort settings
# reviewed monthly; A/B evidence linked in each entry
policies:
- class: customer-email-classification
effort: "off"
evidence: "ab-2026-06-14 (dcost 3.8x, dquality 0.00)"
- class: invoice-json-extraction
effort: "off"
evidence: "ab-2026-06-14 (dcost 4.1x, dquality -0.01)"
- class: contract-clause-classification
effort: "low"
evidence: "ab-2026-06-21 (dcost 1.6x, dquality +0.06)"
- class: coding-agent-step
effort: "medium"
evidence: "ab-2026-06-28 (dcost 3.1x, dquality +0.14)"
- class: multi-hop-reasoning-audit
effort: "medium"
evidence: "ab-2026-06-14 (dcost 3.8x, dquality +0.18)"
- class: fraud-triage
effort: "high"
evidence: "ab-2026-07-05 (dcost 5.2x, dquality +0.22)"
default_effort: "off" # fail closed on cost, not on capability
Three properties are load-bearing. The default is off, not medium — new task classes start cheap and only escalate when a measurement earns it. The evidence field is required — an entry without a linked A/B is a guess, and guesses have a way of aging into "the way we've always done it." The policy is versioned — when a new model tier ships, replay the same A/Bs and update the settings. Vendors change how effort maps to hidden tokens between releases, and a policy that was calibrated against last quarter's model can silently drift into "expensive and unnecessary" or "cheap and wrong" within a month.
The framing to carry away: effort is a per-task-class knob, not a per-application default. The teams that get their reasoning bill under control in 2026 aren't the ones who picked the right tier — they're the ones who picked a different tier for each class and can point to the measurement that justifies each choice. Everyone else is paying the "default medium" tax and calling it the cost of doing business.