Voice is now the interface most agents will spend the most time in, and four platforms have made architecturally opposite bets on how to wire speech, language, and tool-use into one round-trip. The right pick depends less on TTS voice quality than on whether you control the audio path, the model, or just the prompt. ElevenLabs sells voice and rents you a loop on top; Vapi exposes the whole STT-LLM-TTS pipeline as a programmable stack; Retell ships a turnkey contact-center agent; OpenAI's gpt-realtime collapses the pipeline into a single audio-native model. As of late June 2026, picking one is picking which layer of that stack you want to own.
At a glance
Four platforms, four answers to the same question: at which layer of the voice stack does your application live? The table below sets the basics; the chart and matrix show where each one bets hardest.
| Platform | Approach | Self-hosted? | Latency profile |
|---|---|---|---|
| ElevenLabs Conversational AI | TTS-first hosted agent with pluggable LLM, fixed in-house voice | No — managed SaaS | Sub-second turn response, dominated by external LLM hop |
| Vapi | Orchestrator over your choice of STT / LLM / TTS providers | No (cloud) — open-source SDKs available | Sub-second, set by the slowest provider you wire in |
| Retell AI | End-to-end voice agent platform with built-in turn-taking | No — managed SaaS | Sub-second, tuned for high-volume call-center workloads |
OpenAI gpt-realtime |
Audio-native single model — speech in, speech out, no pipeline | No — API only | Lowest of the four; no STT/TTS hops, one model round-trip |
Snapshot: 2026-06-23. Voice latency and pricing for hosted services move fast — verify against each vendor's current docs before sizing a deployment.
ElevenLabs Conversational AI
TTS-grade voice quality as the moat
ElevenLabs entered the voice-agent space from above: a TTS engine and voice-cloning ecosystem that has spent years optimizing prosody, breathing, and emotional inflection. Conversational AI bolts the conversation loop onto that engine. If the deciding factor in your build-vs-buy is whether the bot sounds like a human, this is the platform whose moat actually points at that problem. Voice cloning, multilingual coverage, and a mature library of curated voices come with the product rather than being a separate integration.
Pluggable LLM, fixed TTS
The model layer is configurable: you can point Conversational AI at gpt-4o, a Claude model, Gemini, or an open-weights model behind an OpenAI-compatible endpoint. The TTS layer is not — you are using ElevenLabs voices, which is the whole point. This asymmetry inverts the Vapi posture: with ElevenLabs you swap the brain to chase quality or cost; with Vapi you swap any layer; with gpt-realtime you swap nothing.
Knowledge base and tool-use built in
Conversational AI ships with a hosted knowledge base (upload PDFs, URLs, or text) and a tool-use surface that lets the configured LLM call your webhooks mid-conversation. For a web-embedded voice assistant grounded in product documentation, you can stand up a working agent without writing an orchestration layer — knowledge retrieval, the function-call schema, and the audio path are all the vendor's problem. The trade-off is the usual hosted one: your knowledge base and transcripts live in ElevenLabs' boundary, which matters for any deployment with data-residency obligations.
Vapi
Orchestration-first architecture
Vapi treats voice as a stack to be assembled rather than a service to be consumed. You pick a speech-to-text provider, an LLM, and a TTS provider, and Vapi runs the orchestrator that streams audio between them, manages turn-taking, handles interruption, and wires the result to a phone number or a web client. The framing is closer to a build-your-own voice runtime than a finished product — the source of both its appeal and its operational weight.
Pluggable STT / LLM / TTS stack
Every layer is a configuration choice. Deepgram or AssemblyAI or Whisper for transcription; gpt-4o, Claude, Gemini, or an open model for reasoning; ElevenLabs, Azure, PlayHT, or Cartesia for TTS. For teams that already have opinions about each layer — a Deepgram contract, a Claude prompt library, a cloned ElevenLabs voice — Vapi is the only platform of the four that lets you keep all three. The cost is that you own the latency budget; the round-trip is the sum of your worst-tuned hop.
Programmable phone numbers and dev-friendly pricing
Programmable phone numbers are a first-class primitive: provision a number, point it at a Vapi assistant, and inbound calls flow into your pipeline. Outbound dialing is symmetric. Pricing is a per-minute orchestration fee stacked on top of the per-minute or per-token costs of every provider you plug in — transparent in a way bundled offerings are not, because every line item is yours to see. The bottom line is the sum you control, not a brochure number.
Retell AI
End-to-end voice agent platform
Retell goes the other direction from Vapi: instead of exposing the stack, it absorbs it. You describe an agent — prompts, tools, allowed transfer targets — and Retell handles the audio pipeline, the model selection, the turn management, and the telephony as a single product. The configuration surface is shaped for a specific kind of application (a phone-answering voice agent for support, scheduling, or qualification), not a generic voice runtime. Where Vapi asks "which providers," Retell asks "which workflows."
Built-in turn-taking and barge-in
Two pieces of voice-agent plumbing that look trivial on paper and consume weeks in practice are turn-taking (knowing when the user has finished speaking versus paused mid-thought) and barge-in (cleanly handling the case where the user starts speaking while the agent is still talking). Retell ships both as defaults tuned for real-world phone conditions — long pauses, cross-talk, DTMF tones. The trade-off mirrors the rest of the platform: behavior that is hard to replicate, but no knob to tune it for a use case the defaults were not designed around.
Contact-center positioning
Retell's market is unambiguous: contact centers running thousands of concurrent calls, with CRM integration, post-call analytics, and warm-transfer to human agents as core features. The product earns its keep when the alternative is a Twilio + custom-orchestration build that takes a quarter to ship and another to harden. For a single web-embedded assistant grounded in your knowledge base, Retell is over-engineered; for a 24/7 phone front door to a business, it is the shortest path from zero to production.
OpenAI gpt-realtime
Audio-native single model (GA August 2025)
OpenAI moved gpt-realtime to general availability in August 2025, and the architectural claim is the most aggressive of the four: there is no STT and no TTS in the pipeline. Audio bytes go into a single model that reasons over speech directly and emits audio bytes back. A conventional pipeline pays for transcription, then reasoning, then synthesis, with a network hop and a buffering step at each boundary; collapsing all three into one model collapses the budget. The trade-off is that you commit to OpenAI's voices, reasoning, and pricing as a bundle — there is no layer to swap.
SIP integration and pricing
The Realtime API speaks SIP directly, so a phone provider can stream call audio straight into the model without an intermediate transcoder. Pricing is per-token over the audio stream — roughly $32 per million input audio tokens at the August 2025 launch, with output and text tokens billed separately. Per-token framing is unusual for a voice product and worth modeling: a long agent monologue bills against output audio tokens in a way a per-minute price would not. The economic profile is closer to LLM-as-a-service than to a managed voice platform.
No orchestration layer
What you do not get with gpt-realtime is a turn-taking strategy, a barge-in handler, a knowledge-base retriever, or a phone number — these are your problem. The Realtime API exposes a WebSocket and a function-call interface; the rest of the application is yours to build. For teams that already operate the surrounding plumbing — a phone provider, a vector store, a tool registry — this is the lightest-weight integration of the four. For teams that wanted the rest of the platform too, it is a building block, not a product.
Cross-cutting comparison
Where audio lives
The four divide along a single axis: whose stack does the audio pass through. ElevenLabs and Retell own the pipeline end-to-end — bytes enter their boundary at the client, traverse their stack, and exit as synthesized speech, with you steering only at the prompt and tool layer. Vapi inverts this: the audio passes through STT and TTS providers you chose under Vapi's orchestration, and Vapi is the conductor, not the instrument. gpt-realtime short-circuits the question — there is no pipeline because there is no pipeline; one model holds the audio for a round-trip and hands it back. The practical consequence is data residency and observability: ElevenLabs and Retell give you finished sound and a transcript; Vapi gives you every intermediate stream because you wired them; gpt-realtime gives you whatever OpenAI chooses to expose at the API boundary.
Model choice freedom
Three of the four let you pick the LLM and one deliberately does not. Vapi is the most permissive, treating the LLM as just another configurable hop — gpt-4o, Claude, an open model behind an OpenAI-compatible endpoint, whatever clears your budget. ElevenLabs is permissive within the bounds of "models that stream cleanly into our TTS"; first-class support for gpt-4o, Claude, and Gemini, with awkwardness around non-streaming endpoints. Retell exposes a configured menu tuned for its turn-taking and barge-in behavior — you pick from a list, you do not wire anything you like. gpt-realtime is the model; there is no other LLM in the loop because the model is the loop. The relevant question is not "is swap supported" but whether you want to bet OpenAI's audio-native model stays at the voice frontier for the life of your product.
Telephony integration
Telephony is where the architectural bets become operational ones. Vapi ships programmable phone numbers as a primitive — provision, point, accept — which makes it the fastest of the four to put on a phone if you do not already have a number. Retell goes further, baking phone-side concerns (warm transfer, DTMF handling, multi-agent routing) into the product, because the product is a contact-center agent. ElevenLabs supports SIP and Twilio integration, but telephony is a route into Conversational AI, not the center of the product — the docs read as an integration guide, not a phone-first design. gpt-realtime speaks SIP directly to the model, the cleanest technical path from a phone provider to the LLM but the one that leaves every higher-level concern (queueing, routing, transcript archival) for you to build.
Per-minute economics
The bill arrives via four different routes. ElevenLabs and Retell publish per-minute rates with the LLM cost folded in; the line item is predictable and surprise risk is bounded. Vapi adds a per-minute orchestration fee on top of every provider's metered cost; the bill is transparent because every component is itemized, but modeling it is your job. gpt-realtime bills per audio token in both directions (roughly $32 per million input audio tokens at the August 2025 GA snapshot) — the most fine-grained model, and the one most likely to surprise teams that mentally price voice in minutes. The cheapest at scale depends on conversation shape: short balanced turns favor per-token; long agent monologues favor per-minute; high-volume contact-center workloads favor whichever vendor will negotiate.
When to pick which
| Use case | Pick ElevenLabs if… | Pick Vapi if… | Pick Retell if… | Pick OpenAI gpt-realtime if… |
|---|---|---|---|---|
| Consumer voice assistant | Voice quality is the differentiator and you want a cloned brand voice. | You want to mix providers (Deepgram + Claude + a cloned voice) under one orchestrator. | You want an end-to-end agent with turn-taking, barge-in, and a single bill. | You will ship with OpenAI's voices for the lowest latency and simplest integration. |
| Contact center / high call volume | Workable for web-embedded support; a built-for-purpose platform is the stronger pick at scale. | You want telephony separate from voice and pay per-minute orchestration on top. | Built-for-purpose contact center primitives — call routing, agent handoff, post-call analytics ship in the box. | You already operate queueing, routing, and CRM glue and want the model as a building block. |
| Phone-only IVR replacement | SIP and Twilio are supported but telephony is not the product's center of gravity. | You want a programmable phone number in minutes with an assistant attached. | Telephony is core; the agent and the phone number ship together with native turn-taking. | Your phone provider can speak SIP directly to the model and you build the rest. |
| Web embed with knowledge base | You want hosted knowledge base, hosted tools, and a high-quality voice in one product. | You want to keep your existing retrieval stack and only rent the audio orchestration. | You want the platform to own agent logic but bring your own knowledge through tool calls. | You will write retrieval as a function the model calls — the KB is your problem. |
| Lowest possible latency | The TTS hop adds a measurable budget even when streamed token-by-token. | Latency is the sum of your hops — tunable, but never as short as no hops. | Comparable to Vapi: tuned for sub-second turn-taking, but still a pipeline. | Audio-native single-model: the shortest round-trip of the four, by design. |
| Highest voice quality | The product's center of gravity — cloned voices, multilingual prosody, mature library. | You can pick the same TTS ElevenLabs uses, plus another for redundancy. | Standard provider voices; quality is fine but not the differentiator. | Voice quality is improving but not yet the dimension the model is optimized for first. |
FAQ
Can I run any of these self-hosted?
None of the four ship a self-hosted runtime as a first-class product. ElevenLabs, Retell, and OpenAI's Realtime API are managed services accessed over the network. Vapi is the closest to a self-host story — its SDKs and orchestrator components are partly open source — but the production path is still the hosted cloud. If a fully on-prem voice agent is a hard requirement, look at the open-source projects in the final FAQ (LiveKit Agents, Pipecat), not any of the four compared here.
Which is cheapest at scale?
It depends on the shape of the conversation. gpt-realtime's per-token billing rewards short, balanced turns; long agent monologues bill heavily because output audio tokens accumulate. ElevenLabs and Retell publish per-minute rates with model cost folded in — predictable and favoring long-form interactions. Vapi adds a per-minute orchestration fee on top of each provider's metered cost, which is often cheapest if you can negotiate provider rates at volume. Model the cost against your actual transcripts before signing anything.
Does gpt-realtime support function calling?
Yes. The Realtime API exposes function calling over the same WebSocket as the audio stream — the model decides mid-conversation to call a function you registered, you execute it server-side, and you stream the result back into the session. This is how you wire knowledge retrieval, database lookups, or external API calls into a voice flow — and the primary mechanism for grounding a gpt-realtime agent, since there is no built-in knowledge base.
How do I handle barge-in / interruption?
Retell and ElevenLabs ship barge-in handling tuned for real-world conditions as a default — when the user starts speaking, the agent stops talking. Vapi exposes barge-in as a configurable orchestrator behavior, which means you tune it but you also debug it. gpt-realtime emits interruption signals over the WebSocket — your code stops playback and feeds the new audio in. The hosted platforms hide this work; the model API exposes it.
Which one ships with telephony out of the box?
Vapi and Retell are the strongest — both treat phone numbers as a first-class primitive provisioned inside the product. Retell goes further into call-center concerns (warm transfer, multi-agent routing, DTMF). ElevenLabs supports SIP and Twilio as documented integrations rather than core features. gpt-realtime speaks SIP directly, which is technically clean but leaves every higher-level telephony concern for you to assemble.
What about open-source voice agent stacks (LiveKit Agents, Pipecat)?
Both are credible if self-hosting is non-negotiable. LiveKit Agents builds on the LiveKit WebRTC stack — a Python framework for assembling STT, LLM, and TTS providers under your own orchestrator, closest in spirit to a self-hostable Vapi. Pipecat is similarly a Python orchestration framework for real-time voice and video agents. Neither replaces the polish of the four managed platforms here, but both are the right starting point if data residency, vendor independence, or full control of the audio path are hard constraints.
Further reading
On this wiki:
- Modalities — how speech sits next to text and vision in modern model APIs, and why audio-native models like
gpt-realtimecollapse a pipeline that was three services a year ago. - The Agent Loop — the perceive-decide-act cycle every voice agent wraps, with audio as the perceive-and-act surface instead of text.
- Cost, Quality, Latency — the three-way trade-off that decides whether you ship per-minute, per-token, or per-hop, and how voice intensifies all three axes.
Project sources:
- ElevenLabs Conversational AI — product page, pluggable-LLM list, knowledge-base and tool-use docs.
- Vapi — orchestrator docs, provider matrix (STT / LLM / TTS), programmable phone numbers, per-minute pricing.
- Retell AI — end-to-end voice agent platform, turn-taking and barge-in defaults, contact-center integrations.
- OpenAI Realtime API docs —
gpt-realtimearchitecture, August 2025 GA notes, SIP integration, per-audio-token pricing. - LiveKit Agents — open-source voice-agent framework on top of the LiveKit WebRTC stack.
- Pipecat — open-source orchestration framework for real-time voice and video agents.