Your voice agent almost certainly already knows when it has written down the wrong email address — and on the only public benchmark that measures the next step, it fixes between a quarter and a third of the errors it caught. Detection is the part teams built and the part that works. Repair is the part nobody scored, and it is where the call is lost.
At a glance
τ-Entity is the spoken-entity benchmark Sierra published in late October 2026, from the preprint circulated in September as τ-Elicitation: Benchmarking multi-turn entity extraction in voice agents. It does one narrow thing, which is why its numbers are readable.
| Property | τ-Entity | Why it matters |
|---|---|---|
| Task | Collect one exact missing value on an outbound callback | One field per call, cooperative caller — no ambiguity to hide behind |
| Size | 200 tasks, 10 entity types | Names, phone numbers, email addresses: the fields that gate everything else |
| Metric | Robust Pass3 — success in all three environment realizations | Reliability, not capability; one lucky run does not count |
| Scoring | Direct comparison of the final database state | No LLM judge, so no judge drift to argue about |
| Text control | A matched text agent passes every task | The gap is the channel, not the model's competence at the task |
| Voice result | Four configurations between 0.14 and 0.41 robust exact success | The best of them fails three calls in five |
The control condition is the whole argument
Benchmarks that report one number invite you to argue about the number. This one reports a pair, and the pair closes the usual escape routes. Hand the same agent the same two hundred tasks over text and it gets all two hundred right. Put the identical task behind an audio channel and the best configuration manages 0.41, the worst 0.14.
So this is not a reasoning failure, and it is not a prompt you can fix. It is also not, despite the obvious guess, mostly a transcription failure in the sense that a better speech model would close it. The paper's own conclusion names the bottleneck as strategy selection and successful recovery — which is to say, the two things that happen after the first mis-hearing, not the mis-hearing itself.
That reading is reinforced by which realism knobs moved the score and which did not. Spelling variations and restarts — a caller saying “no wait, let me start again” — did not detectably change exact success. Mispronunciation increased repair effort. The perturbations that make the audio harder to hear once are survivable; the perturbation that forces the agent to go round again is the expensive one.
Detection is not repair, and the gap between them is the product
Here is the finding worth rearranging a roadmap over. Agents in this benchmark verify a great many of their wrong values. They then repair only 24 to 37 percent of the errors they verified.
Read that twice, because it inverts the standard mental model. The industry's answer to spoken-entity error is confirmation: read the value back, ask the caller to confirm, and treat a confirmed value as correct. That design assumes the hard part is noticing. The measurement says noticing is comparatively easy and the hard part is what the agent does in the forty seconds afterwards.
The shape of that chart is the argument. Going from no repair step to one recovers more than twice as many values — 10 percent to 24 or 26 — so the first increment of ceremony is worth a great deal. Stacking the second mechanism on top of the first adds nine more points, not another sixteen. The two mechanisms overlap because they fight the same error, and the errors neither of them fixes are not errors of evidence. They are errors the agent re-commits.
Sierra's write-up names the two failure shapes directly: sometimes the agent repeats the same mistake, and sometimes it replaces the original value with a new incorrect one. The second is the worse outcome and the one that breaks the usual escalation logic, because a system that produces a different wrong answer on retry looks, to every counter you have, like a system that is making progress.
Why the retry loop reproduces the error
An agent that mis-hears “M” as “N” and then asks the caller to repeat the value has changed nothing about the conditions that produced the error. Same channel, same speaker, same phoneme pair, same decoder. The prior that favoured N over M the first time still favours it. What the caller does change, unprompted, is their delivery: people re-read an identifier differently the second time, faster or with different grouping, which is how a retry produces a new error in a new position rather than a correction in the old one.
This is why the numbers reward different ceremony rather than more of it. Spelling alone and read-back alone score within two points of each other because each supplies one new kind of evidence. Together they score higher because the evidence is of two kinds. A second read-back of the same value supplies no new kind of evidence at all, and the benchmark is quietly telling you it will buy you almost nothing.
The operational rule that falls out is narrow and testable: on a repair attempt, never re-run the strategy that just failed. If the capture was conversational, the repair should be character-level. If the first repair was a read-back, the second should change the evidence type — spelling with a disambiguation alphabet, chunked confirmation, or abandoning free capture entirely and resolving against a candidate set you already hold. The wiki's standing recommendation on that last move is on alphanumerics over voice, and τ-Entity is the strongest evidence yet for it: the benchmark's own best configuration still fails roughly two calls in three, so the cheapest reliable identifier is the one that never crosses the audio channel.
The scaffold works and it bills in seconds
The paper also measures the full treatment: a scaffold that prescribes spelling, read-back, correction and confirmation rather than leaving strategy to the agent. It raises robust Pass3 by 14 to 31 points, and it costs 21 to 28 seconds per call.
Both halves of that sentence are findings. The first says prescription beats discretion — the model's own choice of when to spell and when to confirm is a significant part of what is broken, which is the “strategy selection” the conclusion names. The second says the fix has a unit price, and in voice the unit price is time, because the entire stack below the model meters on the clock.
Most teams read 21 to 28 extra seconds as an obvious cost and a reason to hesitate. Priced properly it is usually the cheap option. On the per-minute rates the hosted voice stacks published through October 2026, half a minute of call time runs single-digit cents; a wrong email address on an outbound callback costs a failed authentication, a second call, and a human. We work that arithmetic out in full on the cost of a voice minute. The short version: the confirmation turn is one of the few quality interventions in agent engineering whose price you can compute exactly, and it is small.
If you take one change from this benchmark, make it an instrumented repair counter rather than a prompt edit. Record, per call: whether a value was ever flagged, which repair strategy ran, and whether the final database value matched. Teams that measure capture accuracy and confirmation rate are measuring the two stages that already work.
What this changes about voice evaluation
Three things, in descending order of how much work they are.
The metric comes first. τ-Entity scores robust Pass3 — all three environment realizations — which is the voice analogue of the passk argument that has been reshaping agent evaluation generally. A single-run pass rate on a cooperative caller is a capability claim, and nobody buys capability on an outbound callback; they buy the value being right. The general case is on eval variance and statistical power.
The scoring method comes second, and it is the part worth copying. The benchmark compares the final database state and therefore needs no LLM judge. Voice evaluation has drifted toward judged transcripts because transcripts are what the tooling returns, and a judged transcript cannot distinguish “the agent said the right thing” from “the right thing got written down.” If your voice agent's job ends in a write, score the write. More on building that kind of suite in evaluating voice agents.
The third is a budgeting change. If repair is the binding constraint, then the latency budget for a call has to carry a repair allowance rather than being optimised to its floor, and the two goals pull against each other inside the same call. That tension is the subject of the latency budget — response latency is a within-turn number and call duration is an across-call number, and a design that wins on one can lose on the other without anybody noticing which lever moved.
What not to conclude
Two guardrails on the headline.
This is a preprint, measured on one task family: one missing field, a cooperative caller, most calls under two minutes. It is deliberately the easiest version of the problem, which strengthens the pessimistic reading of the score but says nothing directly about longer or adversarial calls. And the four voice configurations are a snapshot of a market that ships monthly; the ranking among them is the part of this that will age first, while the capture-detect-repair decomposition is the part that will not.
Nor does it say that speech recognition stopped mattering. It says recognition is no longer the only place to spend, and that for teams who have already turned on the obvious recognition fixes, the next marginal engineer is better placed on the repair path than on another vendor evaluation.
FAQ
What is τ-Entity?
A 200-task benchmark that measures whether a voice agent can collect one exact value — a name, a phone number, an email address — during an outbound callback, recognise when it may have got it wrong, and repair the mistake. It was published by Sierra in late October 2026 and appeared earlier as the τ-Elicitation preprint.
Why does a text agent score 100% on the same tasks?
Because the tasks are not hard as reasoning problems. The text control exists to isolate the channel: it shows that everything lost in the voice condition is lost in audio capture, verification and repair rather than in the model's understanding of what it was asked to do.
What is robust Pass3?
A task counts as passed only if the agent succeeds in all three environment realizations. It measures reliability rather than capability, in the same way passk does for text agents: one success out of three is not a product.
Does adding a read-back fix the problem?
It helps a lot and finishes nothing. Over 347 captures that started out wrong, read-back alone recovered 26% and spelling alone 24%; running both recovered 35%. The remaining errors are ones the agent detects and then fails to correct, often by repeating the same mistake or substituting a different wrong value.
Is the 21–28 second scaffold worth it?
In most deployments, yes. It buys 14 to 31 points of robust Pass3, and at the per-minute rates published for hosted voice stacks in October 2026 the added call time costs single-digit cents, against a failed authentication and a human callback on the other side.
Should we just stop asking callers to read identifiers aloud?
Where you can, yes — resolving against the small candidate set you already hold beats transcribing an arbitrary string, and that remains the highest-leverage fix. The repair path matters for the residual cases where the set really is unbounded.
Further reading
On this wiki:
- Alphanumerics over voice — why exact match is the product of ten coin flips, and how to stop putting the string through the channel.
- The latency budget — where the sub-second turn goes, and what a repair turn costs inside it.
- The cost of a voice minute — pricing the seconds a confirmation turn spends.
- Evaluating voice agents — building a suite that scores the write rather than the transcript.
- Uncertainty and calibration — what it means for an agent to know it might be wrong.
Sources:
- Sierra — τ-Entity: Can voice agents get the details right?
- τ-Elicitation: Benchmarking multi-turn entity extraction in voice agents (arXiv:2609.13602)
- sierra-research/tau2-bench — the implementation, where τ-Entity runs in the intake domain.