The Cost of Being Wrong

Three cost lines everyone has, and the fourth nobody does.

Open any agent business case and you will find inference spend, the platform and engineering line, and — if the team is careful — the cost of human review. All three are invoices. They arrive monthly, they are attributable, and they get optimised because they are visible. See unit economics for how they are usually assembled.

The fourth line is what it costs when the agent is wrong and nobody catches it. It has no invoice. It arrives as a credit memo, a re-run, a customer who does not renew, a regulator's letter, a quarter of reporting built on a number that was never right. Because it is never on the same page as the token bill, the two are never traded off — which is the whole failure.

• It is not the same as review cost. Review cost is what you pay to catch errors. Error cost is what you pay for the ones review missed, and the two move in opposite directions — that is the trade the number exists to price. Cost of human review is one half of a fraction whose other half is usually blank.
• It is not a risk-register entry. A qualitative "high / medium / low" cannot be compared against a dollar figure, so in practice it loses every argument against a cost saving that can be.
• It is not covered by accuracy. "94% accurate" is a rate. The bill is a rate multiplied by three other things, and the other three vary far more across task classes than the rate does.

Decompose it into four terms you can actually source.

Expected error cost per task is a product, and writing it as one is most of the value — because it shows immediately that the accuracy number everybody argues about is one of four factors, and rarely the one with the widest spread.

expected error cost per task
  = P(wrong)              # error rate on this task class
  x P(escapes | wrong)    # review and guardrails miss it
  x unit cost per escape  # what one escaped error costs to fix
  x amplification         # how far it travelled before detection

• P(wrong) comes from your eval set, per task class, not overall. A single blended accuracy number averages a class where errors cost nothing with a class where they cost a great deal, and the average is meaningless for both.
• P(escapes | wrong) is the one teams assume and never measure. It is not one minus your reviewer's diligence; it is strongly correlated with plausibility, because the errors that escape are exactly the ones that looked right. Measure it by seeding known-bad items into the review queue.
• Unit cost per escape is the fully loaded remediation: the fix, the notification, the credit, the engineering hours, the write-up. See Step 3 — you already have these numbers.
• Amplification is the multiplier for errors that propagate before anyone notices. A wrong number that got cited in three downstream reports does not cost three times as much; it costs the price of re-deriving three reports plus the credibility of every number in them.

Source the unit cost from incidents you already had.

The objection to all of this is that the numbers are unknowable. They are not — they are unpooled. Every organisation running the process manually has been paying for human errors in the same process for years, and the accounting for it exists under other names.

• Start from the pre-agent baseline. Refunds and goodwill credits, rework hours, warranty and adjustment reserves, the ticket reopen rate, whatever your finance team books as an operational loss. These are real, audited numbers about the same failure with a different operator.
• Ask what one bad case cost, on ten real ones. Ten reconstructed incidents beat any model. Include the parts people leave out: the meeting, the apology, the manual reconciliation of everything downstream, the two weeks the process was suspended.
• Price detection separately from remediation. Finding out which of 40,000 records were affected is frequently more expensive than fixing them, and it is the part that scales with volume rather than with error rate. Repairing agent side effects is the operational half of this line.
• Do not adjust the human baseline downward for the agent. The tempting assumption is that agent errors are cheaper because they are caught in software. The opposite holds at volume: an agent makes correlated errors, fast, so a single bad pattern produces a thousand instances of the same mistake before anyone reviews the second one. Human error is retail; agent error is wholesale.

Detection latency is a multiplier, not a footnote.

Two deployments with identical error rates can differ tenfold in error cost, and the difference is how long a wrong output survives before someone notices. Latency is the term you have the most engineering leverage over, and it is almost never on the dashboard next to accuracy.

• Measure time-to-detection as a distribution, not a mean. Same-shift detection is a correction; next-quarter detection is a restatement. The p95 is the number that decides which of those you are budgeting for.
• Reversibility caps the multiplier. An action you can undo has an amplification near one, whatever the error rate. Anything that leaves your boundary — money moved, an email sent, a record filed with a third party — cannot be undone and starts compounding immediately.
• Buy latency directly. Delayed execution windows, batched external effects, and a nightly reconciliation against a source of truth are cheap and they collapse amplification. This is usually a far better spend than the same money on model capability.
• Instrument for the reverse lookup. When you find one bad output, the question is immediately "which other outputs share this cause?" and answering it in an hour instead of a week is a tracing decision made long beforehand — see tracing and observability.

The number sets autonomy, and it prices the reviewer.

Once expected error cost is a figure per task class, the two decisions teams argue about hardest stop being matters of taste. Both become arithmetic against the same quantity.

• Review is worth buying when it removes more expected error cost than it costs. A reviewer at $6 a task who catches 70% of a $400 expected error cost is obviously worth it; the same reviewer on a $2 task class is burning money to feel safe. This is the calculation that lets you staff review unevenly, which is the only way to staff it affordably.
• Autonomy is per action, priced. The autonomy ladder is usually argued on comfort; it should be argued on whether the expected error cost of an action exceeds the cost of gating it. That produces a mixed policy — autonomous on the long tail of cheap actions, gated on a handful of expensive ones — rather than a global setting that is wrong at both ends.
• Model comparison changes shape. A model that costs three times as much per task and halves the error rate is trivially worth it when expected error cost dominates the token bill, which is most of the time. Only this framing makes that argument in a currency finance accepts.
• Abstention becomes purchasable. Deferring costs you a review; a confident wrong answer costs you the escape. Whenever the second exceeds the first, you should be paying for abstention explicitly in the rubric — see uncertainty and calibration for the threshold and the generator–verifier gap for why the checker, not the model, sets the ceiling.

Put it on the same page as the token bill, and re-derive it quarterly.

A number computed once in a business case is a slide. The version that changes decisions sits next to cost per successful task, is broken out by task class, and moves when the system moves.

• One row per task class, four columns. Volume, expected error cost per task, inference plus review cost per task, and the ratio. The ratio is the column people read: anything above one is a task class where you are optimising the wrong bill.
• Recompute after every material change. A model swap, a prompt change, a new tool, a policy change to the review queue. Error cost is not a property of the task, it is a property of the current configuration, and quality regression detection is what tells you it moved.
• Track escapes as a first-class counter. Every confirmed escaped error gets logged with its remediation cost, from the same taxonomy your triage uses. After a quarter you are estimating from your own data instead of from the pre-agent baseline, and the estimate gets sharply better — failure taxonomy and triage is where the categories come from.
• Watch the tail, not the mean. Expected error cost is heavy-tailed for the same reason spend is: one escape in a high-consequence class can exceed a year of the average. Carry a p95 per class alongside the mean, the way forecasting agent spend does for cost.