The Cost of Human Review

Put the reviewer in the unit cost, at the wage you actually pay.

Agent cost models are written by the people who watch the token meter, so they contain the token bill and stop. Then someone asks why the savings never showed up in the department's budget, and the answer is sitting in a review queue nobody costed.

The arithmetic is not subtle once you write it down. Take a task an agent completes for $0.40 of tokens, reviewed by a specialist on a fully-loaded $75/hour:

• Six minutes of review is $7.50. The model is 5% of the delivered cost of that task and the human is 95%. Halving the token price changes the unit cost by 2.5%; halving the review time changes it by 47%.
• The reviewer is usually senior to the doer. Agents tend to displace the junior-grade production step and leave the senior-grade checking step, so the average wage attached to a task can go up even as headcount goes down. If a $45/hour analyst used to do the work end to end and a $110/hour manager now reviews the agent's version, you have moved work to a more expensive person.
• Review is paid on every output, success or failure. The cleanup cost of a bad answer is paid on the error rate; the checking cost is paid on 100% of outputs. That single distinction is why the two costs behave completely differently as quality improves.
• Queue time is a cost even though nobody is working. An output that waits nine hours for a reviewer has a nine-hour cycle time, and in any workflow where the result feeds a downstream deadline that latency is the thing the customer experiences. It is also, per async agent UX, where the context needed to review well decays.

Why a better model does not reduce your review bill.

This is the counter-intuitive part and the one that decides whether an agent programme pays back. Teams plan on the assumption that as accuracy climbs from 85% to 95%, the human cost falls with the error rate. It does not, because the reviewer cannot tell which outputs are the 95% without reading them.

• Checking cost is a function of output size and verifiability, not of correctness. A 40-line patch takes about as long to read whether it is right or wrong. The error rate governs how often you also pay the repair cost on top; it does not govern the read.
• Higher accuracy can make review slower per item. When errors are rare they are also less expected, and a reviewer who has seen twenty consecutive good outputs is a worse detector than one who has seen five bad ones. You lose the cheap heuristic — obvious garbage — and are left with the expensive one: actually verifying.
• The residual errors get harder, not easier. The failures a strong model has left are the plausible, well-formed, internally consistent ones. They cost more reviewer-minutes each and are more likely to survive review, which is the mechanism behind the silent-failure tax in where the economics breaks.
• So the savings curve flattens exactly where the marketing curve steepens. Going from 70% to 85% correct is worth a great deal, because it removes repair work. Going from 92% to 96% saves you a little repair and no checking at all, and the checking was the expensive half.

The practical form of this: track review minutes per completed task as a first-class operating metric, next to cost per successful task from agent unit economics. If it is flat across two model upgrades, you have learned that your review process is insensitive to quality — which is information about your process, not your model.

Make the output cheap to check — it is the lever you control.

Before trying to remove review, make it fast. The same work presented two ways can differ by a factor of five in review time, and the difference is entirely in how much of the verification the agent did on the reviewer's behalf.

• Ship diffs, not documents. A reviewer checking "what changed" reads a fraction of what a reviewer checking "is this whole artifact right" reads. Where a task genuinely produces a new artifact, produce it against a template so the novel spans are visually obvious.
• Attach the evidence inline, at the claim. A cited sentence is checkable in seconds; the same sentence with a bibliography at the end is checkable in minutes. Grounding is not only an accuracy technique, it is the single biggest review-time lever available — see hallucination and grounding.
• Cut the unit of work down. Review time rises faster than linearly with output size, because holding a large artifact in working memory is the expensive part. Four small reviewed items beat one large one even at the same total volume.
• Structure what can be structured. Fields a machine can validate should never reach a human. Schema-valid output, arithmetic that reconciles, links that resolve, dates in range — burn all of it in a pre-check so the reviewer spends their attention on judgement rather than proofreading. See structured outputs.
• Make the agent say what it is unsure about, specifically. "I could not confirm the second figure against the source" directs the reviewer's eye and is worth more than a global confidence score. It also fails safe: if the flag is wrong the reviewer reads more, not less.
• Buy back review with reversibility. An action that can be undone in one click can be reviewed after the fact by exception rather than before the fact in full. This is the cheapest review-cost reduction in existence and it is an engineering decision, not a model one — the mechanics are in undo and reversibility.

Selective review is the only thing that actually removes the cost.

Everything in step 3 makes the reviewer faster. Only one thing makes the reviewer disappear from a share of the traffic, and it is not accuracy — it is a defensible rule for which outputs go out unread.

• You need a routing signal, and it has to be calibrated. A model's stated confidence is not it; the alignment step that made the model agreeable also made it confident. Sampling agreement across several runs, retrieval-support checks, and validator outcomes are the signals that actually separate — see uncertainty and calibration.
• Set the threshold off a coverage–risk curve, not a hunch. Score a labelled sample, sort by the signal, and read off what fraction you can auto-approve at a chosen error rate. "We auto-approve the top 60% and the measured error rate in that slice is 0.8%" is an operating decision. "The model seems good now" is not.
• Keep a random audit running on the auto-approved slice, forever. Sample a few percent and review them anyway. This is what converts your threshold from a one-off study into a live control, and it is the only way you find out the slice has drifted before a customer does. Budget it — it is small, and it is not optional.
• Tier by consequence rather than by confidence alone. Irreversible, externally-visible, or regulated actions stay in full review at any confidence. The savings come from the high-volume reversible middle, which is usually most of the traffic. The placement rule is the one in human in the loop.
• Expect the first threshold to be conservative and plan to move it. Auto-approving 30% of traffic removes 30% of the largest cost line in the system. That is a bigger win than any model change on the table, and it compounds every time the audit lets you widen the band.

The costs review hides, and the ones it creates.

A review step is not a free safety net. It has its own failure modes, and two of them make the economics worse while appearing on no ledger.

• Rubber-stamping is the default steady state. A reviewer approving mostly-correct outputs at volume converges on approving by default, at which point you are paying the full review cost for approximately none of the detection. Measure it directly: the reject rate and the time-per-review both trending down together is the signature, and it is the exact failure the approval UX in approval and confirmation UX is designed to resist.
• The reviewer is a hard scaling ceiling. Agent throughput is elastic and reviewer throughput is not. A system that produces work faster than it can be checked converts into a queue, and queues turn a throughput win into a latency loss with no change in the cost model. If you cannot hire review capacity linearly, autonomy is capped by staffing rather than by capability.
• Review quality is a function of load. The tenth review of the hour is not the first. Any cost model that treats reviewer-minutes as fungible overestimates the detection you are buying at high volume.
• Attention has an opportunity cost you are already paying. The senior person reviewing agent output was doing something else. Costing them at wage understates it — the real number is what the review displaced, which is usually the highest-value work in the team.
• The review record is also a liability. Approvals are attributed to a named person and retained. That is exactly what you want for accountability, and it is also a discoverable record — worth deciding deliberately rather than by observability default, per retention and legal hold.

Instrument it, then price the decision.

None of this is actionable without measurement, and review time is the one input almost nobody logs. Three numbers make the whole model live.

• Review minutes per completed task. Timestamp when an item enters review and when it is resolved, and subtract idle. Approximate is fine; absent is not. This is the numerator of everything above.
• The review ratio — review minutes over the minutes the task would have taken unaided. Under about 0.3 the agent is clearly winning. Around 0.5 you are buying speed and consistency rather than cost. Above roughly 0.7, once you add coordination and queue latency, you are running a more expensive process with extra failure modes, and the honest move is to narrow the scope until the ratio falls rather than to keep tuning prompts.
• Auto-approved share, with the audited error rate beside it. One number is the saving, the other is the licence to keep it. Report them together or the first will drift up until something breaks.

Then attribute the cost like any other. Review minutes belong in the per-task cost view in cost attribution and budgets, tagged by workflow, so that when a team proposes a new agent surface the reviewing capacity it consumes is part of the proposal rather than a surprise for someone else's headcount.

Related: agent unit economics for the denominator this fits into, measuring agent ROI for the counterfactual, uncertainty and calibration for the routing signal, and designing for trust for the interface that makes a review worth its minutes.