Accessible Agent Interfaces

7 min read

H19
Playbook · Agent UX & Human Interaction

Accessible agent interfaces.

An agent UI can pass every automated accessibility check and still be unusable with a screen reader, because nothing that breaks here is markup — it is timing. Assistive technology assumes a page that settles; an agent produces a page that never does, and then puts the one control that actually matters, the approval, behind a countdown. Fix the temporal problems and the checklist ones take care of themselves; fix only the checklist and you ship a green audit on an interface nobody can drive.

STEP 1

The failures are temporal, which is why your audit missed them.

Automated tooling checks a snapshot: contrast ratios, label associations, alt text, heading order, focus outlines. Every one of those is worth passing, and none of them describes what goes wrong in an agent interface.

  • A token stream appends to the DOM dozens of times a second. If that region is a live region, the screen reader either floods — reading fragments over the user's own navigation — or coalesces unpredictably and drops the middle of the answer. Both are unusable; only one is obvious in a demo.
  • The page has no completion event that assistive technology can observe. A sighted user sees the cursor stop. A screen-reader user has no equivalent signal unless you build one.
  • Controls appear during the run. Focus is somewhere; a new "Approve" button materialises somewhere else; nothing tells the user it exists, and by the time they find it the state has changed underneath.
  • The output shape is different every run, so nothing about the interface can be learned. This is the cost of generative UI that gets discussed least.

The single most common defect: wiring the streaming answer itself to aria-live. It is the intuitive move — new content, announce it — and it produces an interface that talks over its own user for forty seconds. Never make the token stream a live region.

STEP 2

Announce at semantic boundaries, not token boundaries.

Split the surface into two regions with different jobs, because they have different temporal behaviour.

  • A status regionaria-live="polite", updated once per meaningful event and never more than roughly once every few seconds. "Searching the codebase." "Found 12 files." "Waiting for your approval." This is the narration, and it should be terse enough that a person can follow it while doing something else.
  • The answer region — not a live region at all. It is a document. Mark it with a role and a label, announce once that it is complete and how long it is, and let the user navigate it with their own tools, at their own pace, in their own reading order.

The completion signal is the part that gets skipped and the part that costs the most. Without it, the user cannot distinguish "still thinking" from "finished" from "crashed", so they poll — arrow into the region, read, arrow back, wait, repeat. Announce completion explicitly in the status region, and put the token count or line count in that announcement so they can decide whether to read it now.

Use aria-live="assertive" for exactly one class of event: something that requires the user to act, right now, or that reports a failure they need to know about. If everything is assertive, the interface is shouting, and users turn it off.

STEP 3

The approval dialog is a safety feature that becomes a barrier.

This is where the interests genuinely collide. Approval UX wants friction, urgency and a clear diff; accessibility wants time, stable focus and content that survives being read linearly. Get the collision wrong in the obvious direction and the person who most needs to review the action is the one who cannot.

  • Move focus to the dialog, and trap it there. An approval that appears without taking focus does not exist for a keyboard user. Return focus to the invoking control when it closes.
  • Never put an irreversible action on a timer. Auto-approve-after-30-seconds is a decision made by a stopwatch, and it discriminates precisely against the users who read most slowly. If you need a default for unattended runs, make the default decline, and say so in the dialog.
  • Make the diff readable, not just visible. Colour-only diffs convey nothing to a screen reader and little to a colour-blind user. Add a text prefix per line, a count ("3 additions, 1 deletion"), and a plain-language summary of the consequence above the diff — which is better for everyone and is the thing sighted users read anyway.
  • Say what happens if nothing happens. "This will not run until you approve" is a sentence the interface owes the user, and it is what makes it safe to take the time to read.

Auto-approval timers are the accessibility defect most likely to be defended as a feature. If the argument is "otherwise runs stall", the answer is asynchronous handoff — see async agent UX — not a countdown that silently converts a human decision into a timeout.

STEP 4

Anything with a clock in it needs an escape hatch.

Agent interfaces are full of implicit timing assumptions that nobody wrote down as requirements.

  • Interruption windows. If "stop" only works while a spinner is visible, users who need longer to locate the control cannot stop the agent. Keep abort available for the whole run and for a grace period after it.
  • Voice barge-in. A voice agent tuned to interrupt on the first syllable will cut off a user with a speech disability or a slower speaking rate mid-sentence. Endpointing thresholds are an accessibility setting; expose them.
  • Session and idle timeouts. A long-running agent task plus an idle timeout equals lost work for anyone who steps away or works slowly. Warn before expiry and offer an extension.
  • Auto-scroll. Following the stream fights with a user who has scrolled up to read. Stop auto-scrolling the moment the user moves, and offer an explicit "jump to latest".
  • Motion. Honour prefers-reduced-motion on typing effects, shimmer placeholders and progress animation. A blinking cursor effect for forty seconds is a vestibular problem, not a delight.
STEP 5

Non-determinism is a cognitive-accessibility problem.

Every conventional interface can be learned. The control is in the same place tomorrow, the flow has the same number of steps, and expertise accumulates. An agent interface can violate all three on consecutive runs, and the cost of that lands hardest on users with cognitive disabilities, users with memory impairments, and — quietly — on everyone under time pressure.

The mitigation is to hold the frame still even when the content moves:

  • Fixed landmarks. Status, output, actions and history live in the same regions every run, with the same labels, even when a region is empty. An empty labelled region is more usable than a region that sometimes does not exist.
  • Stable action vocabulary. The same operation gets the same verb every time. Do not let the model name the buttons — that is the shortcut generative UI offers and it destroys learnability.
  • Predictable ordering. Actions in a consistent order, with the destructive one never in the position the safe one occupied last run.
  • A plain-language mode. One toggle that drops the reasoning trace and shows what the agent did and what it wants, in short sentences. Useful far beyond its intended audience, which is the usual test of whether an accessibility feature was designed or bolted on.
STEP 6

Test it the way it fails: over time, with the actual tool.

Static analysis cannot catch any of the above, so a green axe report tells you almost nothing here. The tests that find real defects are behavioural and they are cheap to run once you decide to:

  • Run one real task end-to-end with a screen reader — NVDA or VoiceOver, monitor off. Not a static page: a full run with streaming, a tool call, an approval and an undo. Most teams find three blocking defects in the first fifteen minutes, and the first one is usually the live region.
  • Complete an approve-and-undo cycle keyboard-only. If you cannot reach the approval, read the diff, decline it, and recover, the flow is broken regardless of what the audit says.
  • Snapshot the announcement sequence as a fixture. Record what gets announced during a scripted run and assert on it in CI. It is the only regression test that catches someone making the answer region live again six months from now.
  • Test with a slow run and a failing run. Ninety seconds of latency and a mid-run tool error are where the status region either earns its place or reveals that it only ever narrated the happy path.

If you do one thing: take the streaming answer out of aria-live, add a terse status region that announces at semantic boundaries, and announce completion explicitly. That single change converts the most common agent interface from unusable to navigable with a screen reader, and it takes an afternoon. Then go delete your auto-approval timer — it is the other defect that a checklist will never flag and a real user will hit on day one.