Appointment scheduling over voice: the defect is in the write, not the listen.
A voice agent that books appointments will double-book before it mishears anything, because the ten to fifteen seconds between offering a slot and hearing “yes” is a window in which the calendar moves. Scheduling is a concurrency problem wearing a speech-recognition costume: hold one slot before you speak it, confirm it with a conditional write, and have the recovery sentence written before you need it.
Step 1Measure the gap between the offer and the acceptance.
Every scheduling turn has the same shape: the agent reads availability, speaks one or more options, the caller decides, the agent writes. Three of those four steps take real time, and nothing holds the inventory during them.
Count it out. “I have Tuesday the eleventh at two fifteen, Wednesday the twelfth at nine thirty, or Thursday at four o'clock” is about twenty-four words — roughly 9.6 seconds at a natural 150 words per minute. Add the caller's deliberation, which is two to five seconds when they are choosing between three times and longer when they are checking their own calendar, plus 500–800 ms for end-of-turn detection and the round trip to write. The realistic window from read availability to write accepted is 12 to 18 seconds, and none of it is under your control once the agent starts talking.
Now put volume against it. A line handling 40 scheduling offers an hour has, in any given 15-second window, an expected 40 × 15 / 3600 = 0.17 other live offers — about one in six. That sounds survivable until you notice the distribution: every agent on every line offers the earliest available slots first, because that is what callers want and what every prompt tells the model to do. The overlapping offers are not spread across the calendar. They are concentrated on the same one or two slots, so the one-in-six collision rate is close to the double-booking rate rather than being diluted by it.
This failure does not show up in an ASR evaluation, a barge-in test, or a latency budget, which is why teams ship it. It shows up as “the agent booked someone into a taken slot” in a support ticket, and gets misfiled as a hallucination.
Step 2Reserve before you speak — and reserve exactly one slot.
The fix is to take the slot off the shelf before the words leave the agent's mouth, not after the caller agrees. A tentative reservation with a short time-to-live converts a race into a lease: whoever holds the lease can confirm, everyone else is told the truth immediately instead of ten seconds later.
The instinct is to hold every option you are about to offer. Resist it, because holds are inventory you have removed from circulation, and at scale the holds themselves cause the shortage. For a busy line at 400 offers an hour with a 20-second TTL:
- Holding three options per call — 400 × 3 × 20 / 3600 = 6.7 slots held at any instant. Against a same-day pool with eight slots left, two thirds of the remaining capacity is invisible to everyone, including your web booking page.
- Holding one option per call — 400 × 1 × 20 / 3600 = 2.2 slots. The same protection on the slot that matters, at a third of the cost.
So hold the one you are recommending — the first option, the one most callers take — and offer the others unheld, as candidates you will have to re-check. That asymmetry is honest and it is cheap: the recommended slot is protected, and the alternatives carry the risk they always carried.
Three properties make a hold safe rather than a new leak: a TTL on the order of 20–30 seconds (one utterance plus one turn, not one call); release on hangup, on topic change, and on confirmation of a different slot; and a per-call cap so a long, meandering call cannot accumulate holds. Treat an un-released hold as a resource leak with a dashboard, in the sense of durable state and resumability.
Step 3Confirm with a conditional write, never a blind one.
A hold narrows the window; it does not close it. Holds expire, a human at the front desk can book over you, and a second agent on another line may have a hold the first system never learned about. The write itself has to be the thing that fails safely, which means it carries a precondition describing the world the agent spoke about.
The HTTP mechanism is standard and widely implemented: read the resource with its entity tag, then write with If-Match set to that tag. If the resource changed, the server returns 412 Precondition Failed and your write does not land. Google Calendar's v3 API carries an etag on every event resource, and most booking back-ends expose either an etag or a version column you can use the same way. Pair it with an idempotency key so that a retry after a timeout cannot produce a second appointment — the classic double-booking bug is not two callers, it is one caller and one retried request.
# Offer -> hold -> speak -> confirm. The precondition is the whole design. hold = calendar.hold(slot=best, ttl_s=25, call_id=call.id) if not hold.granted: best = next_candidate() # never speak an unheld recommendation speak(f"I can do {best.spoken}. Does that work?") answer = listen() try: booking = calendar.confirm( hold_id=hold.id, if_match=hold.etag, # 412 if the world moved idempotency_key=f"{call.id}:{best.id}", ) except PreconditionFailed: # expired or taken: STEP 4 recover(call, reason="slot_gone") else: speak(f"Booked: {booking.spoken}. You'll get a text.")
Note what the if not hold.granted branch prevents. Without it the agent speaks a slot it does not have, which is the single highest-cost sentence in the whole interaction — the caller now believes something false, and every later correction reads as incompetence rather than as diligence.
Step 4When the hold expires mid-sentence, have the sentence ready.
A 412 on confirm is not an error to log and retry silently. It is a conversational event, and the agent has roughly one turn to handle it before the caller decides the system is broken. Write the recovery utterance in advance and test it, because a model improvising under a tool failure will either apologise at length or — much worse — book something else and describe it as what the caller asked for.
The shape that works names the change, offers the nearest alternative, and asks one question: “That one just went — the next I have is Tuesday at three. Shall I take it?” Three properties matter. It is one sentence, so the caller does not lose the thread. It contains exactly one new time, not a fresh list. And it does not explain the concurrency model, which no caller wants.
Two failure modes to prohibit explicitly in the system prompt and verify in evaluation:
- Silent substitution. Booking an adjacent slot and reporting success. This is the one that generates complaints and, in healthcare and legal contexts, real harm; it is a goal-drift failure with a tool result as the trigger.
- Unbounded retry. Looping on confirm while the caller waits in silence. Cap at one automatic re-attempt within the same turn, then speak. The deadline discipline is the same one in timeouts and deadline budgets, except that here the budget is measured in how long a human will tolerate dead air — about two seconds.
Where the caller cannot be served at all, the handoff is a warm transfer that carries the attempted slot with it, so the human does not restart the conversation.
Step 5Ask the discriminator before you offer anything.
The timing arithmetic in STEP 1 has a second implication: the longer the offer, the wider the window. Reading three options costs about ten seconds of exposure; reading one costs three. So the cheapest way to shrink the race is to ask one narrowing question first and then offer a single held slot.
The discriminator is almost always a day part or a day, because that is what callers actually have constraints about: “Mornings or afternoons?” then “I can do Wednesday at nine thirty — does that work?” Two short turns beat one long one on every axis that matters here — exposure window, recall, and the recency bias that makes callers accept whichever option they heard last. The general case is in long lists over voice; scheduling is its most common instance and the one where the cost of the list is a wrong booking rather than a confused caller.
Two rules that follow directly:
- Never speak an index. “Press or say one for Tuesday” invites a caller to say “one” when they meant the first thing they remembered, which is usually the last thing you said.
- Speak times the way the caller will repeat them. “Half past nine on Wednesday” is confirmable; “09:30 on 2026-11-11” is not. The spelling-out discipline is in alphanumerics over voice.
Step 6Instrument the three failures that look like one.
“The agent booked it wrong” is three different defects with three different fixes, and an aggregate booking-accuracy number hides which one you have. Emit all three separately:
- Hold-expiry rate — holds that timed out before confirm, as a fraction of holds granted. Rising means your offers are too long or your TTL is too short. This is the metric that tells you STEP 5 is not being followed.
- Precondition-failure rate — 412s on confirm, as a fraction of confirms attempted. Rising means genuine contention: more concurrency than the pool supports, or a channel writing without holds (the front desk, the web form, a second agent).
- Mismatch rate — appointments whose booked time differs from the time the transcript shows the agent speaking. This is the only one that catches silent substitution, and it requires joining the booking record to the transcript rather than trusting the agent's own report of success.
The end-to-end check that catches everything the three metrics miss is reconciliation: after each call, re-read the calendar and compare what is there against what the agent said. Disagreement is a defect regardless of which component caused it — the same “check the effect, not the intent” principle as replay testing with recorded traces, and the reason evaluating voice agents has to include a write-side assertion and not only a transcript score.
Do this today, before any prompt work: take one week of bookings, join each to its call transcript, and count the rows where the spoken time and the booked time disagree. That number is your real scheduling accuracy. If it is not zero, the fix is a hold and an If-Match — not a better model, not a longer prompt, and not more examples.