Every text-to-speech vendor selling to voice agents leads with time-to-first-audio, and the numbers on those pages — 40 ms, 75 ms, sub-90, sub-200 — are lab conditions. Independent measurement of the same cloud endpoints puts the medians between roughly 188 and 313 ms, and the figure that decides whether a phone call feels human is not that median but the spread around it: one vendor in this comparison ships four times the jitter of another at a better median. Pick on the tail, and the tail is something you buy with deployment rather than with a model name.
At a glance
Four vendors that all target real-time conversational agents, with what each advertises and where each will let you run it.
| Vendor | Real-time model | Advertised time-to-first-audio | Where it runs |
|---|---|---|---|
| ElevenLabs | Flash v2.5 / Turbo v2.5 | ~75 ms (Flash) | Cloud and VPC via SageMaker or Vertex; on-prem and on-device in early access |
| Cartesia | Sonic (state-space architecture) | sub-90 ms | Cloud, on-device, and on-prem in early access at a seven-figure commit |
| Deepgram | Aura-2 | sub-200 ms | Cloud, VPC, and self-hosted containers |
| Rime | Mist v3 / Coda | ~40 ms p90 (self-hosted Mist v3) | Cloud, VPC, and on-prem containers, generally available |
Take the measured numbers as a shape rather than a scoreboard: they are point-in-time, region-dependent and load-dependent, and every vendor here ships new models on a monthly cadence. The shape is what matters, and it is consistent across every independent harness published this year — the marketing number is a floor achieved under ideal conditions, real medians land two to seven times higher, and the ordering by median is not the ordering by consistency.
The tail is the product
A voice turn is a budget, not a benchmark. Roughly 300 ms of silence after a caller stops speaking is the point at which the exchange stops feeling like a conversation, and the TTS engine is spending its share of that budget after endpointing, after the model's first token, and before a jitter buffer that most media stacks hold at 200 to 400 ms.
That composition is why the interquartile range matters more than the median here. A run at the median is a call that sounds fine. A run at the 95th percentile is a caller who starts talking again because they assumed the line dropped — and every one of those is a barge-in, a discarded response, and a turn that now has to be repaired. Compare the two shapes in this field: a model at 188 ms median with 100 ms of spread will regularly deliver turns worse than a model at 264 ms median with 28 ms of spread, and the second one never surprises the caller. Consistency is what a listener actually perceives, which is the same reason agent latency is a p95 discipline everywhere else too.
The corollary is uncomfortable for anyone shopping on the front page of a pricing site: the median is the number that is advertised, discussed and benchmarked, and it is the wrong number. Ask a vendor for a p95 and p99 at your concurrency, in your region, over a week. A vendor that cannot produce one is telling you something.
Barge-in: you are billed for speech nobody heard
Interruption is the normal case in a real deployment, not an edge case — callers cut agents off constantly, and a good agent invites it. When they do, the pipeline has to stop playback, flush a buffer holding a few hundred milliseconds of audio, cancel the in-flight synthesis and switch the microphone back to turn-taking. Two consequences fall out that nobody puts on a comparison chart.
The first is behavioural: your effective TTS latency on an interrupted turn is not first-audio time, it is cancel-to-silence time, and the streaming protocol decides it. Cancellation over an open WebSocket is fast; a REST-shaped integration that has to wait for a response to finish is not, and the caller hears the agent talk over them, which is the single most damaging failure a voice product has.
The second is commercial: character-based billing charges you for text you submitted, and an interrupted turn means a share of that text became audio nobody heard. Whether you pay for it is a contract detail that no benchmark measures — some vendors bill on characters submitted, some on characters synthesised before cancellation. At a realistic interruption rate on long agent turns this is not a rounding error, and it is the first question to ask in a pricing conversation, not the last.
Design around it rather than negotiating around it: shorter agent turns are cheaper, more interruptible and better UX simultaneously. An agent that says one sentence and stops wastes less synthesis, recovers faster from barge-in, and gives the caller a turn boundary they can use.
Price moves about 2.5×, and predicts nothing
List pricing for the latency-optimised tiers of these four clusters in a narrow band — roughly $0.02 to $0.05 per thousand characters, with volume tiers shaving that and premium or multilingual tiers on the high-fidelity models sitting well above it. Across the wider market the published range runs from about $4 to $200 per million characters, and it is worth knowing where in that range your provider sits before you argue about milliseconds.
What the spread does not do is predict either latency or consistency. The cheapest per character in this group is not the slowest; the most expensive is neither the fastest nor the most consistent. What price buys is voice quality, cloning, language coverage and prosody control — which are real purchases, just not latency purchases. If your agent reads addresses, order numbers and dollar amounts, the feature that saves you is entity-aware normalisation, and Deepgram's Aura-2 markets exactly that; if your product needs a recognisable brand voice in twenty languages, that is ElevenLabs' territory and the premium is the point.
Run the arithmetic on your own traffic before this becomes a debate: characters per minute of speech is roughly stable across vendors, so a per-minute figure converts cleanly. At that point TTS is usually a minority of a voice agent's per-minute cost — telephony, STT and the model itself take more — which is a good argument for choosing on tail latency and deployment, and a bad argument for choosing on the character rate.
Deployment is how you actually buy the tail
Everything above points at one lever. Network variance, noisy-neighbour effects and regional routing produce most of the spread in a cloud endpoint's distribution, and the way to remove them is to move the synthesis next to your media server. All four vendors now have some non-cloud answer, so the checkbox question — does it offer on-prem? — separates nobody. The question that separates them is how far down-market that path reaches.
- Rime has made self-hosting the default posture rather than an enterprise concession — on-prem is generally available, ships as containers you run on your own cloud or metal, and the company reports that the majority of its volume runs that way. Its advertised ~40 ms p90 is a self-hosted number, which is the honest way to quote one.
- Deepgram has offered self-hosted deployment for years across cloud, VPC and on-prem, with published, unusually specific hardware requirements — Aura-2 wants exactly two dedicated GPUs per TTS engine container, a constraint you want to know before capacity planning, and a level of detail that suggests the path is well travelled.
- Cartesia offers on-prem in early access with a seven-figure minimum commitment, plus an on-device story that its state-space architecture makes unusually credible. The option exists; whether it exists for you depends on your budget.
- ElevenLabs now runs in your own cloud through SageMaker and Vertex, and announced on-premises and on-device deployment in April 2026 — aimed explicitly at organisations that cannot procure cloud infrastructure in their region, and shipped as early access rather than a container you pull. The path exists; it goes through a sales conversation.
So the practical test is not whether a vendor has an on-prem page. It is whether you can be running it next quarter without an early-access queue or a seven-figure commitment — and on that test the field splits two and two. Self-hosting is not free — you inherit GPU capacity planning, model upgrades and an on-call rotation, the same trade laid out in self-hosted inference for agents. It is simply the only lever that moves the tail by an order of magnitude rather than a few percent.
When to pick which
| Situation | Lean | Because |
|---|---|---|
| High-volume telephony, latency is the product | Rime or Deepgram, self-hosted | Containers next to your media server remove the network variance that dominates the tail |
| Regulated audio that cannot leave your network | Deepgram or Rime | Both ship generally available self-hosted deployments today, without an early-access negotiation |
| Brand voice, many languages, quality is the product | ElevenLabs | The voice catalogue and cloning are what the premium buys, and nothing else here matches it |
| Fastest cloud median with engineering to absorb jitter | Cartesia | Best measured median in the group, provided your buffer strategy tolerates the spread |
| Reading identifiers, addresses, amounts aloud | Deepgram | Entity-aware normalisation is a correctness feature, not a cosmetic one |
| On-device or embedded | Cartesia | The state-space architecture is the one built for it |
FAQ
Why is the measured latency so far above the advertised number?
Advertised time-to-first-audio is typically measured at the model boundary under ideal conditions — short text, warm connection, co-located client, no load. Your number includes TLS setup or a pooled connection, queueing at the provider, network transit from your region, and a first chunk large enough to be useful. None of that is dishonest; it just means the published figure is a floor rather than an expectation.
Is a 100 ms interquartile range actually bad if the median is lowest?
It depends entirely on your buffer. If your media stack already holds 300 ms of jitter buffer, spread inside that window is absorbed and the median wins. If you have tuned the buffer down to chase responsiveness, spread lands directly in the caller's ear as inconsistent pauses, and a tighter distribution at a worse median is the better call.
Should I benchmark this myself?
Yes, and it is a day of work rather than a project. Run each candidate from your own region, at your real concurrency, with your real utterance-length distribution, for at least a full business day, and record p50, p95 and p99 for time-to-first-audio and for cancel-to-silence. Vendors change models monthly; any published table including this one ages.
Does streaming the LLM output into TTS fix the latency problem?
It helps a lot and it changes what you should measure. Sentence-level streaming hides model generation time behind the first spoken clause, so the metric that matters becomes time-to-first-audio plus the reliability of chunk delivery, not end-to-end synthesis. It also makes cancellation harder, because more of the response is already in flight when the caller interrupts.
Where does TTS sit in overall voice agent cost?
Usually a minority share. Telephony minutes, speech-to-text and the model behind the conversation typically add up to more, which is why a 2.5× spread in character pricing rarely decides the architecture. Model the whole per-minute stack — the latency budget and the cost budget are the two numbers a voice product actually runs on.
Further reading
On this wiki:
- The speech stack — how STT, the model and TTS compose into one turn.
- Latency budget — where the 300 ms goes.
- Turn-taking and barge-in — the interruption path in detail.
- Evaluating voice agents — measuring the thing callers actually experience.
- Deepgram vs AssemblyAI vs ElevenLabs vs Speechmatics — the other half of the stack.