Provisioned Throughput & Commitments

Two instruments, routinely conflated.

"We're going to commit and get a better rate" collapses two different contracts that fail in different ways. Separate them before you model anything, because one is a capacity reservation and the other is a volume discount, and only the first one changes your latency.

• Provisioned capacity reserves dedicated serving throughput for you: Azure OpenAI sells Provisioned Throughput Units, AWS Bedrock sells Provisioned Throughput on one- and six-month terms, Google Vertex AI sells Generative AI Scale Units. You pay for the reservation whether or not you use it. It is a rental of machines expressed in throughput units, and the units are model-specific in ways that make cross-vendor comparison genuinely hard.
• Committed spend is a discount for promising a dollar volume over a term. It does not reserve anything, it does not change your rate limits, and it does not improve your p99 by a millisecond. It is a purchasing instrument, and its risk is under-consumption, not under-utilisation.
• They fail differently, so model them differently. Under-using provisioned capacity wastes money continuously and silently. Under-consuming a spend commitment usually surfaces once, at true-up, as a bill for the shortfall. The first is a daily operational leak; the second is a forecasting error.
• Only one of them is on the critical path of an incident. When the shared pool is saturated, provisioned capacity is why your requests still complete. A spend commitment is a line in a contract that nobody can page.

The break-even is a utilisation number, and it is one division.

Reserved capacity is priced per unit of time; on-demand is priced per token. So the comparison is not "which rate is lower", it is "at what utilisation does the fixed cost per hour equal what those tokens would have cost on demand". Do it yourself, with your own token counts, before anyone quotes you a percentage.

• The rule of thumb is the complement of the discount. If reserved capacity is quoted at 30% below on-demand at full utilisation, you break even at 70% sustained utilisation. At 40% off you break even at 60%. Vendors currently advertise discounts across roughly that band depending on term length, so your break-even lives somewhere in the 60–75% range — a demanding number for any interactive workload.
• Utilisation means "of the reservation, over the whole term". Not peak. Not business hours. If your traffic runs twelve hours a day and drops to nothing overnight, your ceiling is 50% before you have made a single other mistake, and no amount of optimisation inside those twelve hours recovers it.
• The unit is throughput, so input and output are not interchangeable. Reserved capacity is sized on tokens per minute at some assumed input/output ratio and context length. Agent workloads with long prefills and short completions consume a reservation completely differently from chat, and a reservation sized on a chat-shaped assumption will be exhausted at a token volume far below the headline figure.
• Compute it on cost per completed task, not per token. A reservation that lets you route more work to a smaller model changes the numerator and the denominator at once — the metric that survives that is the one in unit economics.
• Caching moves the break-even against the reservation. Prompt caching is often the larger and more reversible saving, and it reduces exactly the token volume you were about to reserve. Do it first, then size. Sizing a reservation on uncached traffic is how teams buy capacity for work they were about to stop doing — see context caching economics.

Agent traffic is close to the worst possible shape for a reservation.

Reserved capacity rewards flat, predictable, well-characterised load. Every structural property of agent workloads pushes the other way, and they compound rather than average out.

• One user action fans out into an unpredictable number of model calls. A task might take three steps or thirty depending on what the tools return. Your load is therefore not a function of user count in any stable way, which is the assumption capacity planning normally rests on.
• Token volume grows super-linearly within a task. Each step re-sends the accumulated transcript, so the cost of a long task grows closer to the square of its step count than to its length — the arithmetic in agent cost control. A modest increase in average task difficulty is a large increase in throughput demand.
• Peaks are correlated, not independent. Batch jobs, business-hours load and retry storms arrive together. The peak-to-mean ratio that determines what you must reserve is set by the correlated worst case, while the utilisation that determines whether it paid off is set by the mean.
• Retries are demand you did not intend to buy. A degraded upstream produces a burst of retries at exactly the moment capacity is tightest, which is where the interaction with rate limits and provider capacity and the queueing discipline in concurrency and scaling matters more than the size of the reservation.
• The honest fix for burstiness is a queue, and it is nearly free. If any part of your load tolerates delay — evaluations, batch enrichment, background agents, indexing — moving it behind a queue that fills the troughs raises utilisation directly. Do that before buying capacity to cover peaks you could have flattened.

The real reason to buy it: a tail you can promise.

Everything above argues against reserved capacity on price. On latency it argues the other way, and this is where the instrument genuinely earns its cost. Shared on-demand endpoints have a tail set by other people's traffic; a reservation replaces someone else's noisy neighbour with your own known load.

• Buy it when a customer-visible p99 is contractual. If a support agent must answer inside a few seconds, or a voice agent inside a turn budget, the variance of a shared pool is the risk, not the mean. Reserved capacity is how you make that number yours to control — the trade-off framing is in cost, quality and latency.
• Buy it when saturation is an availability event. If being throttled during a peak means a queue you cannot drain and users who leave, the reservation is insurance and should be priced as insurance — against the cost of the outage, not against the on-demand rate.
• Buy it when you need an admission-control point you own. With dedicated capacity, the decision about which requests get served under pressure moves inside your system: you can shed batch work, degrade to a smaller model, and protect interactive traffic. On a shared endpoint the provider makes that decision for you, and it does not know which of your requests matter.
• Do not buy it for compliance reasons that do not exist. Dedicated throughput is not usually dedicated hardware, a data-residency control, or a privacy boundary. If someone is justifying the purchase on those grounds, check the contract rather than the marketing page.
• Write the SLO down before you sign. "p95 under 4 seconds at 300 concurrent sessions" is a purchasable thing you can verify after the fact. "Better performance" is not, and it is the justification that survives into the renewal unchallenged.

The term price nobody puts in the model: it freezes your model choice.

This is the term that turns a marginal deal into a bad one, and it never appears in the spreadsheet. Reserved capacity is bought per model. A twelve-month commitment on a specific model is a bet that the model you want to be running in month eleven is the model you chose in month one — in a market where the frontier has been moving on a quarterly cadence and price-performance with it.

• The switching cost becomes financial, not just technical. Without a commitment, moving to a better or cheaper model costs a re-qualification. With one, it also costs the remaining reservation, and that number is large enough to win arguments it should lose.
• Watch for the decision that quietly stops being made. The failure is not a dramatic one. It is a team that skips an evaluation because "we're on the commitment anyway", and a year later is serving a model two generations behind at a rate that stopped being competitive in month four.
• Provider-side deprecation does not respect your term. Models get retired and silently updated on the provider's schedule; your reservation does not protect you from a re-qualification you did not plan. Run the drill in model deprecation and migration regardless of what you have committed to.
• Prefer the shortest term that buys the SLO. A one-month reservation at a smaller discount that you can walk away from is usually a better instrument than a six- or twelve-month one at a better rate, because the optionality you are selling is worth more than the spread in this market.
• Reserve the floor, burst for the rest. Size the reservation to the load that is genuinely always there and let the peaks run on demand. It gives up some headline discount and gives back most of the utilisation risk, and it is the configuration that survives a traffic forecast being wrong in either direction.

The decision procedure, in order.

Run these in sequence. Most teams reach a clear answer at step three and never need the rest, which is the point — the cheapest capacity decision is usually the one you avoid making.

• 1. Get per-workload token accounting first. If you cannot say which workload consumed what, at what input/output ratio and what context length, you cannot size anything. That is what cost attribution is for, and it is a prerequisite rather than a parallel effort.
• 2. Exhaust the free levers. Caching, context discipline, routing cheap steps to smaller models, and queueing the deferrable load. All four reduce or flatten the demand you were about to reserve, and all are reversible.
• 3. Plot a week of real load at one-minute resolution. Read off the mean, the p99 and the ratio between them. If peak-to-mean is above roughly two and you cannot flatten it, reserved capacity sized for the peak will not reach its break-even, and you have your answer.
• 4. State the SLO the reservation buys, or stop. A latency or availability commitment you would actually put in front of a customer. No SLO, no purchase.
• 5. Buy the floor, on the shortest term, with burst enabled. Then measure realised utilisation weekly and treat a sustained figure below break-even as a live incident, not a quarterly finding.
• 6. Re-open the self-hosting question honestly at high, flat volume. At the point where a reservation is well utilised around the clock, the arithmetic in self-hosted inference for agents starts to compete — and it carries an operational burden that a reservation does not.

Related: build vs buy for the larger version of this decision, economics failure modes for the other ways the model goes wrong, and inference providers for who is selling what.