Cost Attribution & Budgets

8 min read

B3
Operation · Economics & ROI

An estimated cost does not just carry error. It carries error that ranks your features in the wrong order.

Two requests with byte-identical prompts can differ more than twelvefold in input cost, because one wrote a cache entry at 1.25× the base input rate and the other read it at 0.1× — and nothing in your prompt tells you which happened. That matters far beyond precision: a busy feature runs warm and a quiet one runs cold, so estimating cost from token counts overstates the first by up to an order of magnitude while understating the second, and the feature you kill is the cheap one. Attribution is therefore a logging decision made at the call site before the first request ships, not an analysis you can perform later, because the fields that decide the number are returned by the provider and reconstructible from nothing else.

Step 1The invoice is aggregated along the two dimensions you cannot act on.

The bill arrives by API key and by month. The questions that drive decisions are which feature is unprofitable, which tenant is subsidised by the others, which user is an outlier, and which code path regressed after Tuesday's deploy. None of those is answerable from the invoice, and no amount of after-the-fact analysis recovers a dimension that was never recorded — which is the whole reason this is an engineering task rather than a finance one.

So attribution means carrying the dimensions you decide on through every call, and recording spend against them as it happens. The part teams get wrong is not the carrying. It is believing that "spend" can be computed later from what they did record.

Step 2Record the provider's usage object, because three line items are not in your prompt.

The intuitive estimate — count the tokens you sent, multiply by the list price — is wrong in three ways that all push in traffic-dependent directions.

  • Cache state. Anthropic bills a cache write at 1.25× the base input rate for the five-minute lifetime and 2× for the one-hour lifetime, and a cache read at 0.1×. The response tells you which you got, in cache_creation_input_tokens and cache_read_input_tokens; your prompt does not, because it is identical either way. OpenAI reports the same distinction as input_tokens_details.cached_tokens.
  • Reasoning tokens. They are billed at the output rate and counted inside output_tokens, itemised under output_tokens_details.reasoning_tokens, and they do not appear in the visible response text. Any estimate derived from the length of what came back undercounts a reasoning-heavy call — and undercounts it worst when the model thought hardest, which is when the call was most expensive. Accounting for reasoning tokens works through the measurement side.
  • Retries and fan-out. One user task is not one request. It is the product of your step limit, your parallel tool calls, your sub-agent fan-out and every retry policy in the stack, and the largest term is in no config file — the model deciding to try again differently. Retry amplification is the full treatment; for attribution the consequence is that spend has to be summed over a request tree you did not enumerate in advance.

The fix is mechanical and it is the single highest-value line in this page: record the usage object the provider returned, keyed to your cost context, on every call. Not your estimate of it. Not the token count you computed before sending. The returned object, stored whole, because next quarter's question will need a field you did not think to extract.

A corollary that catches teams who did everything else right: if your cost figures come from a tokenizer you run locally against your own prompt, they are a model of your spend, not a measurement of it — and the model has no term for cache state, no term for reasoning, and no term for the retries that happened downstream of the call you measured.

Step 3Work the arithmetic once, and the misranking becomes obvious.

Take a shared 8,000-token system prefix — instructions, tool schemas, a few examples — and price it in units of the base input rate, so the numbers transfer to any model.

  • Uncached: 8,000 × 1.0 = 8,000 units.
  • Cold, writing a five-minute cache entry: 8,000 × 1.25 = 10,000 units.
  • Warm, reading it: 8,000 × 0.1 = 800 units.

Cold to warm is a 12.5× spread on the identical prefix. Now apply it to two features sharing that prefix. Feature A serves a request every few seconds, so almost every call reads a warm cache and truly costs about 800 units of prefix. Feature B is used twice an hour, always past the five-minute window, so every call writes and truly costs 10,000. Estimate both at the list input rate and you record 8,000 for each: you have overstated A by a factor of ten and understated B by a fifth. A is roughly twelve times cheaper per call than B on this component, and your dashboard says they are identical.

That is the argument of this page in one worked example. The error in an estimate is not noise that averages out across features; it is a function of call frequency, which is precisely the variable that distinguishes features from one another. So the estimate does not merely blur the ranking, it inverts it — and attribution exists to produce that ranking. A number that misorders the two things you were choosing between is worse than no number, because you will act on it.

While the arithmetic is on the table, two things fall out of it for free. The five-minute cache is ahead of uncached from the first reuse (1.25 + 0.1 = 1.35 against 2.0 for two uncached calls). The one-hour cache, at a 2× write, needs the second reuse to pull ahead (2.0 + 0.1 + 0.1 = 2.2 against 3.0). If your traffic does not reliably deliver a second hit inside the hour, the longer lifetime is a loss — and that is a decision you can only make from recorded hit rates, which brings you back to STEP 2.

Step 4Stamp a cost context on every call and propagate it through fan-out.

A cost-context object is created when a task starts and travels with it, including into every sub-agent and tool call. Each response's usage is recorded against that context. The hard part is propagation, and it is hard in exactly the place the money is: a sub-agent that loses the context attributes its spend to nobody, and fan-out is where the expensive deep work lives.

# cost/attribution.py — context travels with the task, into children
class CostContext:
    def __init__(self, tenant, feature, user, version):
        self.tenant, self.feature = tenant, feature
        self.user, self.version   = user, version

    def record(self, resp):
        u = resp.usage                      # the PROVIDER's object, not an estimate
        emit("spend", tenant=self.tenant, feature=self.feature,
             user=self.user, version=self.version,
             input=u.input_tokens,
             cache_write=u.cache_creation_input_tokens,
             cache_read=u.cache_read_input_tokens,
             output=u.output_tokens,         # includes reasoning tokens
             reasoning=u.reasoning_tokens,
             raw=u)                          # keep it whole; next quarter needs a field

def spawn_child(ctx):
    return ctx          # SAME context — unattributed spend is the bug

Store the token counts rather than a computed dollar figure, and apply prices at query time. Prices change, and a stored dollar amount freezes the rate you happened to be on, which makes a year of history uncomparable and silently misstates the quarter in which a provider cut a price. Price deflation and cost per task is why this matters more in this market than in most.

Then reconcile, because reconciliation is the only thing that tells you the pipeline works. Sum your recorded spend for the month, compare it to the invoice, and publish the gap as a percentage next to every attribution number you report. A gap under a few percent means your contexts propagate; a large one means spend is landing in an unattributed bucket, and that bucket is disproportionately the deep multi-step work, because that is the work that fans out. The honest version of a per-feature cost is the figure plus its reconciliation error, exactly as unit economics argues a unit cost is only as good as the verifier behind it.

Step 5Attribute to the dimension that maps to a lever, then make the budget a runtime control.

Logging the model name is easy and nearly useless. Attribute along axes that map to an action: feature (kill or reprice it), tenant (renegotiate or rate-limit), user cohort (detect abuse, or find a power user to learn from), version (catch the deploy that doubled cost). The test for a useful dimension is that a clear bad reading on it triggers a specific action this week. If nothing would happen, the tag is telemetry theatre. One addition in this market: record the model and effort setting as a dimension too, because a provider default that moves under you changes cost with no diff in your repository — the failure unpinned vendor defaults is about.

And a monthly budget compared to actuals in a dashboard does nothing, because by the time the dashboard is red the money is gone. A real budget is a counter, scoped per tenant or feature, decremented in the request path, that changes behaviour when crossed.

  • Per-tenant ceilings, so one customer's runaway cannot consume the shared pool and starve everyone else.
  • Soft then hard thresholds — at the soft line, degrade (cheaper model, lower effort, smaller context, optional tools off); at the hard line, fail closed and alert.
  • Checked before the expensive call, not reconciled after it.
  • A circuit breaker on velocity, tripping on spend per minute or on cost-per-task exceeding a multiple of the historical median. Alerting narrates an incident already in progress; the breaker bounds it. A system with only alerting has chosen to pay for every incident in full, and with a stolen credential in the path that bill is somebody else's volume — see denial of wallet.

Step 6When finer attribution costs more than it saves.

Per-call attribution and a hot-path budget check add latency, storage and code surface. For a low-volume internal tool on a flat budget, the instrumentation can genuinely cost more than the spend it governs. Attribute at the granularity your decisions use and no finer.

The split that holds up in practice: always record the provider's usage object and always run the per-tenant circuit breaker. The first is a few fields on a log line you are already emitting, and it is the one thing you cannot backfill — a month without it is a month you can never analyse. The second is cheap and bounds catastrophe. Reserve per-user and per-call rollups, and the storage they imply, for products where a pricing or margin decision genuinely depends on them.

If you run a credit-metered product, note that you now have a second denominator — credits consumed versus credits sold — and the two can move in opposite directions. Credit-metered agent products covers that case, and it is the one place where attribution feeds pricing directly rather than through a margin review.

Do the smallest version this week: add the provider's returned usage object, stored whole, to the log line your calls already emit, with tenant, feature and version beside it. That is an afternoon, it is the only part of this page that cannot be done retrospectively, and it is what makes every later question answerable. Then reconcile one month against the invoice and write the gap down — if it is large, fix context propagation through sub-agents before you build a single dashboard, because a chart over an unattributed majority is a confident wrong answer.