After-call work: the summary outlives the call by years, and nobody evaluates it.
The caller hears the conversation once; the disposition code and the summary your agent writes are read for the next three years — by the next agent that picks up this customer, by the router that decides where their callback goes, by the analyst counting refund reasons, and eventually by whoever is answering a regulator. A voice agent can handle every turn beautifully and still poison all of that in the four seconds after the caller hangs up, because after-call work is generation into a durable record with no listener present to correct it. Evaluate the artefacts, not the conversation.
Four artefacts, four different consumers, four different costs of being wrong.
"After-call work" is a single label over outputs that fail in unrelated ways, and treating them as one generation task is the root error. Separate them before you build anything:
- The narrative summary. Read by the next human who touches this account. Wrong is expensive but recoverable, because a human reading it has the transcript one click away.
- The disposition or wrap-up code. A single value from a closed vocabulary that drives routing, reporting, compensation and sometimes regulatory counts. Wrong is silent and compounding — nobody re-reads a disposition, they aggregate it.
- Structured field writes. Address changed, plan upgraded, callback scheduled, consent flag set. These are side effects, not text, and they inherit every hazard in voice tooling and state.
- Follow-up tasks. A ticket, a callback, an escalation to a queue. Wrong here usually means an obligation to the customer that exists in the transcript and nowhere else.
Rank them by how easily a downstream human can catch the error, and you get your review budget for free. The summary is self-correcting because it sits next to its own evidence; the disposition is not, because it is consumed as a number. Spend your evaluation effort inversely to how visible the artefact is.
The disposition is a classification problem wearing a generation problem's clothes.
Asking a model to "select the wrap-up code" looks like a small task and is the highest-leverage thing on this page, because the code is the only after-call output that gets counted. Two things go wrong, and only one of them is the model's fault.
- Constrain the output to the vocabulary. A disposition must come from the enumerated set, enforced at the decoding or schema layer rather than requested in the prompt — the standard argument in structured outputs. A near-miss code that does not exist in the CRM is caught; a real code that is merely wrong is not.
- Your taxonomy is probably already broken, and the agent will expose it. Most wrap-up code sets were designed for humans clicking under time pressure, so they contain overlapping categories, a dominant "Other", and codes nobody has used since 2019. Human agents resolved the ambiguity by habit and by team; a model resolves it differently every call, and the resulting distribution shift gets blamed on the model. Audit the taxonomy against a month of real calls before you automate against it.
- Force an abstention. There must be a code that means "the model was not confident", distinct from "Other", and it must route somewhere a human sees it. Without an abstention the model spreads its uncertainty evenly across plausible codes, which is the worst possible failure because it is invisible in aggregate.
- Watch the distribution, not the accuracy. A per-code confusion matrix against human labels tells you where to intervene; a single accuracy number hides that the agent is systematically collapsing three billing codes into one.
The summary must not assert anything the transcript does not support — and the transcript is already wrong.
Two error sources stack here and they compound in one direction. The model can over-claim, filling a plausible gap with an invented commitment; and the transcript it is summarising already contains recognition errors, concentrated exactly on the names, account numbers and amounts that matter most, which is the entity-error problem the speech stack describes. A summary launders both into confident prose that no longer carries any signal that it was uncertain.
- Quote the caller for anything consequential. "Said the charge on the 14th was not hers" is verifiable against the audio; "disputes a fraudulent transaction" is a conclusion the agent reached and a legal characterisation it was not asked to make.
- Mark inference as inference. Anything the agent concluded rather than heard belongs in a separate, visibly labelled part of the record, so the human reading it can tell the difference — the grounding discipline from hallucination and grounding, applied where nobody will check.
- Never generate an entity the agent did not verify. Account numbers, amounts and dates in the summary should come from the system of record or from a confirmed slot, never from the ASR output alone. If the value was never confirmed back to the caller, it does not belong in a durable field.
- Write what was promised, prominently. The single most damaging omission is a commitment made on the call — a callback, a waiver, a deadline — that exists only in the audio. Extract commitments as their own structured output, not as a sentence inside the narrative.
- Keep the transcript, and link it. The summary is a lossy derivative; the record of what was said is the artefact your audit trail actually needs, and the two should never be separable.
Writes are side effects, and the call has already ended.
The dangerous property of after-call work is that it executes when nobody is listening. A mid-call tool failure produces dead air and a caller who complains; a post-call write failure produces silence, and the first symptom is a customer three weeks later insisting they changed their address.
- One idempotency key per call, carried on every write. Retries, duplicate webhooks and a supervisor re-running a failed batch will all fire the same write again, and a second scheduled callback is a customer-visible defect. This is idempotency in its most ordinary form and it is routinely skipped because the call is over.
- Design the partial write. The summary lands, the disposition lands, the field update fails. Decide in advance whether the record is left inconsistent or the whole packet is held for retry, and make the half-written state detectable rather than plausible-looking.
- Handle the transferred call explicitly. When the call was escalated, a human is also writing to this record — see escalation and warm transfer. Agree who owns the disposition, and never let the agent's asynchronous write overwrite a human's after they took over.
- Carry compliance state into the record. Recording consent, disclosure that the caller was speaking to an AI, and any regulatory flags are part of the durable output, not part of the conversation. A call that was compliant while it happened must still be provably compliant a year later.
- Never let after-call work write outside the caller's authorisation. Whatever caller authentication established bounds these writes exactly as it bounded the in-call ones; the write path must not become the place where scope quietly widens.
After-call work has a latency budget too, and it competes with the next call.
Human contact centres measure this and give it a name; agent deployments usually forget it exists, because the seconds are not in the turn latency budget anybody is watching. Three decisions follow:
- Synchronous or asynchronous. Doing the work inline holds the session, the context and the audio, and blocks the line. Doing it asynchronously frees the line and requires you to durably keep everything the summariser needs, including which model and prompt version produced it.
- Asynchronous means a queue, which means a backlog. A writeback queue that falls behind produces a CRM that is minutes or hours stale — and the caller who rings back immediately, which is the highest-value moment to have the record, is exactly the one who hits the gap. Alert on queue age, not queue depth.
- Reconcile. Every completed call must end with a record, and something must notice when one does not. A daily reconciliation of calls against written records is unglamorous and is the only control that catches a writeback path failing silently for a week.
Prefer asynchronous with a short, monitored queue: it is the only shape that lets you re-run after-call work when you improve the summariser. If the artefacts are a pure function of a stored transcript, a prompt version and a taxonomy version, you can regenerate a month of them — which turns a summariser upgrade from a change that only affects future calls into one you can evaluate on the past.
Evaluate the artefacts, because your voice eval does not.
A voice evaluation suite grades the conversation: did the agent understand, respond, resolve. It typically stops at the moment the caller hangs up, which means the outputs with the longest half-life are the only unevaluated part of the system. Extend the golden set past the end of the call:
- Disposition agreement against human labels. Have experienced agents independently code a sample of calls and measure agreement per code. Also measure agreement between your humans — if they disagree with each other, the taxonomy is the defect and no model will fix it.
- Summary faithfulness, scored for assertions not present in the transcript. One unsupported claim is a failure regardless of how good the rest of the paragraph reads; count claims, not quality.
- Commitment recall. Of the promises made on the call, what fraction appears in the record? This is the number a customer feels directly, and it is usually far worse than summary quality suggests.
- Write success and duplicate rate. Operational, boring, and the first thing to break when a CRM schema changes underneath you.
- Downstream repeat rate. When a customer calls back, does the next agent have what they need, or does the caller explain it again? Feed this into evaluating voice agents as the outcome measure for the whole after-call path.
Start with the disposition, because it is one token of output that drives more downstream behaviour than the entire summary and is the cheapest thing on this page to get right. Constrain it to the vocabulary, add an explicit abstention that routes to a human, and check your taxonomy against a month of real calls before blaming the model for a distribution shift. Then make the summary quote rather than characterise, extract commitments as structured output rather than prose, put one idempotency key on every write, and reconcile calls against records daily. The conversation is judged in real time by someone who can push back; everything written afterwards is judged by nobody, which is exactly why it needs the eval.