Where the Economics Breaks

8 min read

B6
Operation · Economics & ROI

Agent economics invert in the denominator, and the largest multiplier is a default nobody chose.

A profitable agent and a money-losing agent are usually the same agent on different inputs, and the flip is invisible on the token meter, because every inversion here moves the denominator rather than the price. Retries, the p99 tail and escalation each multiply spend per finished piece of work while cost per token holds perfectly flat — and the retry multiplier in particular is assembled from library defaults you inherited rather than picked. The only number that shows the inversion is cost per accepted outcome, watched at the p99.

Step 1Count your retry multiplier before you argue about model prices.

Nearly every agent stack retries in at least two layers, and in most of them neither layer was configured. Both of the vendor SDKs that agents are actually built on ship a non-zero retry default: openai-python and anthropic-sdk-python each define DEFAULT_MAX_RETRIES = 2, so a single call you believe you made once is up to three billed attempts before your own code sees an error. Anthropic's client retries on connection errors, 408, 409, 429 and any status at or above 500, and will also obey a non-standard x-should-retry response header when the server sets one. LangChain's ChatOpenAI leaves max_retries unset by default and passes through, so it inherits the SDK's two rather than adding its own — one of the few defaults in this stack that composes politely.

The durable-execution layer is where it stops composing politely. Temporal retries Activities automatically, and its default Retry Policy starts at a 1-second interval with a 2.0 backoff coefficient, caps the interval at 100× the initial one, and sets Maximum Attempts to unlimited — the field defaults to zero, which is evaluated as unlimited. Workflow Executions, by contrast, are not retried by default. So the ordinary shape of a production agent — a Temporal Activity wrapping one model call — is three HTTP attempts inside an unbounded number of Activity attempts, and the only thing standing between that and a runaway is the schedule-to-close timeout, which is not set for you either.

# what the defaults actually compose to, per logical call
sdk_attempts      = 3        # DEFAULT_MAX_RETRIES = 2, plus the original
activity_attempts = "unlimited"   # Temporal maximumAttempts = 0
framework_retry   = 1        # langchain-openai passes through: no extra layer

# and in a nested agent, the multiplier is per level
depth = 5
worst_case(3 ** depth)     # 243 billed calls for one logical step

Read your own numbers off the config rather than off this page — these are the shipped defaults as of November 2026, and vendors move them. The point is the arithmetic: a retry policy is a unit-economics decision wearing a reliability costume, and the worst case is correlated. When a provider degrades, every task in the fleet retries in the same minute, so the multiplier does not arrive spread thinly across a quarter. It arrives as one afternoon that becomes the month's largest cost line. See retry amplification for the failure shape and idempotency and retries for the construction that makes retrying safe at all.

Step 2The long tail eats the margin the median earned.

Task cost is power-law distributed. The median task is cheap; the p99 task is routinely 50–100× it, burning loops, fan-out and re-read context on the inputs the agent handles worst. Because the tail is a small fraction of volume, it is invisible on a mean-cost dashboard the whole time it is consuming most of the spend.

# the mean is healthy; the tail already ate the profit
median_cost = 0.18
p99_cost    = 14.0          # ~78x the median — same agent, hard input
tail_share  = 0.02          # 2% of tasks ...

tail_spend_fraction = (0.02 * 14.0) / (0.98 * 0.18 + 0.02 * 14.0)
# ≈ 0.61  — 2% of tasks are 61% of total cost

The tail is not a separate population of unlucky tasks. It is where everything else on this page lands: the unbounded retry, the tool result nobody truncated, the run that crossed a long-context price tier and doubled the rate on its entire request. That is why capping beats optimising here. A per-task ceiling on steps and tokens, enforced by the harness rather than requested in the prompt, converts an unbounded tail into a bounded one — and a bounded tail can be priced.

Optimising the median optimises the 39% of spend that was never the problem. Cap, route or explicitly refuse the tail, because it will not appear in any average you are watching until the invoice does. Cost attribution is the prerequisite: you cannot find a p99 you cannot join to a run.

Step 3Your break-even escalation rate does not contain the token bill.

Human escalation is the safety valve that quietly destroys the unit economics, and the reason is selection: escalations do not sample the workload, they cluster on the hard cases that were always the slow ones. Work a population rather than an average. Say 80% of tasks take a person 10 minutes and 20% take 40 minutes, so the all-human average is 16 minutes. The agent clears the easy ones and escalates the hard ones — but an escalated task is not a 40-minute task, because the human now has to read the agent's partial work and decide whether to trust it before starting. Call that 25% overhead: 50 minutes.

# minutes of human time per task
all_human      = 0.80 * 10 + 0.20 * 40      # = 16.0
escalated_task = 40 * 1.25                  # = 50.0 (read the agent's work first)

# with the agent, human time = escalation_rate * escalated_task
at_20_percent  = 0.20 * 50                  # = 10.0  -> 37% saved, not 80%
at_35_percent  = 0.35 * 50                  # = 17.5  -> worse than no agent

break_even     = 16.0 / 50.0                # = 0.32

Two things fall out of that arithmetic, and both are counter-intuitive. First, the break-even escalation rate is just the all-human average task time divided by the escalated task time — the agent's token bill does not appear in it at all. At these numbers the deployment inverts somewhere around a 32% escalation rate no matter what you pay per million tokens, which is why "switch to the cheap model" is not a response to this failure mode. Second, because escalation selects the tail, the escalated task time is always above the all-human average, so the break-even rate is always below 50% — often far below. An agent whose escalation rate drifts from 20% to 35% has not got 15% worse. It has crossed from saving a third of the work to costing more than doing nothing.

So publish the break-even rate as a number next to the escalation rate itself, and alert on the ratio rather than the level. The two inputs are measurable: time the escalated tasks, and time a sample of the same work done without the agent. If you have never measured the second one, you do not know which side of the line you are on — see the cost of human review and fractional time savings, where the minutes-saved claim gets taken apart properly.

Step 4The assurance bill scales with autonomy, not with revenue.

Making an autonomous agent trustworthy costs money the pilot never paid: judge calls on sampled or every output, a regression suite re-run on every model and prompt change, trace storage with a retention clock, and human review of the eval set itself because an unmaintained eval silently stops measuring. That cost tracks the agent's autonomy and blast radius, not the revenue it earns — so the safer and more autonomous you make it, the more the assurance line crowds the margin, and it crowds it hardest precisely where the agent is allowed to act without a human in front of it.

Two accounting errors make this invert without warning. Booking it as one-time R&D hides a recurring cost of goods: every model deprecation forces a re-run, and model deprecation is not an event you schedule. And costing review in money while rationing it in people ignores the real constraint — review capacity is a headcount ceiling, not a budget line, so the agent's throughput is capped by the reviewers long before it is capped by spend. That ceiling is the subject of oversight capacity; the spend side is the cost of evaluation.

Step 5The silent-failure tax is the only line that never reaches a meter.

The most expensive failure costs nothing on any dashboard you own: a confident, wrong answer that passes the agent's own checks, ships, and detonates downstream as a wrong refund, a bad merge, a misfiled record. Its cost is not in the model bill. It is the cleanup, the trust damage, and the human re-verification of everything the agent has touched since.

  • It is unmetered. No token counter sees it. It surfaces weeks later as a support cost, a chargeback, or churn, by which time it has no attributable run.
  • It taxes the wins too. One silent failure forces re-checking of the correct outputs as well, which erases the labour saving that justified the agent — and that re-check is the escalation arithmetic in STEP 3 applied retroactively to work you had already booked as done.
  • It compounds the ROI claim. Your ROI counted the output as a success. The real ledger pays for it twice, and the cost of agent errors is where that second payment gets priced.

The operational consequence is that this failure mode is bounded by detection latency, not by error rate. An agent with a 2% silent-error rate and same-day detection is a manageable product; the same agent with a three-week detection lag is a repair project, because by then the wrong actions are a backlog of judgements rather than a set of rows you can roll back. That asymmetry is the whole argument of repairing what the agent already did, and it is why failure concealment is an economics problem and not only a safety one.

Step 6When the honest move is to narrow scope or not ship.

When the tail cannot be bounded, the escalation rate will not stay under its break-even, or the silent-failure tax exceeds the labour saved, the economics do not improve with scale — they invert harder, because every failure mode here is amplified by volume rather than diluted by it. The fix is rarely a cheaper model. It is a narrower scope on which the agent is actually reliable, or a decision not to ship this workflow at all. An agent that is only profitable on its easy inputs is not a product with a tuning problem; it is a smaller product pretending to be a bigger one — and shipping the smaller one deliberately is a better outcome than discovering it from an invoice. Scaling back a deployment is the version of this decision taken late.

Four changes, in order, none of which requires an architecture change. One: set max_retries=0 on the vendor client and own retry in exactly one layer, with maximumAttempts and a schedule-to-close timeout set explicitly rather than inherited. Two: put cost per accepted outcome on the dashboard at p50 and p99, where accepted means it passed your evaluation and not that it returned. Three: compute the break-even escalation rate from measured minutes and alert when the live rate crosses a fixed fraction of it. Four: move eval and review spend out of R&D and into cost of goods, so the margin you report is the margin you have. Related: unit economics for the base model, and price deflation and cost per task for why falling token prices do not rescue any of the five.