Clinical inbox triage agents.
Every vendor in this market ships a drafter, and the largest study of a deployed drafter found that switching it on made message turnaround slower at the clinic level even though individual clinicians who used it were faster. That inversion is the design brief: the burden in a clinical inbox is not composing the reply, it is deciding who has to touch the message at all, so build the router first and measure messages closed without a clinician — not minutes saved per draft.
Step 1The two numbers that should pick your architecture point in opposite directions.
The evidence base here is unusually good for a clinical AI workflow, because the drafting tools shipped early and at scale inside Epic and got studied retrospectively on real traffic. The headline results:
- A cohort study in JAMA Network Open covering 983,154 patient-portal message responses from 1,542 clinicians found that adoption of an AI draft-response tool was associated with an 8.6% increase in message turnaround time, while actually using a draft on a given message was associated with a 29.5% reduction.
- After-hours work did not meaningfully fall. That is the outcome the deployment was justified by, and across the published work it is the outcome that most consistently fails to move.
- An earlier quality-improvement study of 162 clinicians found statistically significant improvement on burden and burnout measures with no change in time spent.
- A pediatric study reported perceived savings of roughly 1.2 minutes per message, with burnout unchanged.
Hold the first and last together, because their product is the argument. A saving of 1.2 minutes a message, if it were real time rather than perceived time, is enormous: at a deliberately conservative 14 messages a day — the figure one large health system published as being among the lowest in the Epic community — across 230 clinical days, that is roughly 3,200 messages and about 64 hours a clinician-year. The time studies do not find 64 hours. They find no change, and at clinic level they find the queue moving 8.6% slower once the tool is available.
The explanation that fits all four results is unglamorous and it is the whole playbook: a draft invites a clinical review of messages that would otherwise have been dispatched in seconds or never reached the clinician at all. The tool makes the hard messages faster and makes the easy ones into reviewable artefacts. Net of both, you have automated the step that was cheap.
Volume is the reason this is worth engineering rather than staffing around. Patient-authored portal messages rose 153% between 2020 and 2025 on national Epic data — from about 2.2 to 5.4 per sending patient per year — while clinician and staff replies rose 24% to 5.70 per patient per year, having peaked at 6.86 during the pandemic. The reply side is already near its ceiling. The arriving side is not.
Step 2Build the router first, and make its output a disposition rather than prose.
The agent's primary output should be a routing decision with a named destination, not text. Five dispositions cover the traffic, and they are ordered by who must be licensed:
- Closed from the record, no human. Results already released and unchanged, appointment logistics, form and letter status, "did you get my fax". The reply is a lookup, and the agent should cite the record field it read.
- Non-clinical pool. Scheduling, billing, insurance, medical records. These need a person and that person is not a clinician — and in most inboxes this is the single largest misrouted class.
- Protocolled clinical pool. Nurse, pharmacist or MA acting under a standing order: a refill of a stable chronic medication inside protocol, a documented symptom that maps to an existing triage algorithm.
- Clinician, routine. Genuinely needs the panel's clinician, not today.
- Clinician, urgent. Needs a clinician on a clock. This is the only class where a routing miss harms a patient, and the entire safety case lives in its recall.
Structuring it this way gives you something a drafter cannot: an objective you can optimise without touching clinical judgement. Three of the five destinations are administrative, and moving a message from "clinician, routine" to "non-clinical pool" removes a clinician touch entirely rather than shortening it. Draft generation then becomes a second, optional stage that runs only on the dispositions where a human will reply anyway — which also means the drafter stops creating review work on messages the router already closed.
Two implementation notes that decide whether this survives. Make the disposition reversible and visible: a message the agent closed from the record should land in a lightweight audit queue rather than vanishing, at least until the class has months of clean history — the shape is a review queue, not a silent action. And make "I cannot tell" a first-class output routed to the urgent-clinician class; an abstention that costs a clinician thirty seconds is the cheapest control in the system, and a router forced to pick among five labels will pick wrong on exactly the messages you most want escalated.
Step 3The safety claim you are shown is per task. Convert it to per clinician-year before you believe it.
On 16 November 2026 OpenAI announced healthcare computer-use agents alongside more than 20 health-tech plugins, and reported that reviewers rated 99% of 325 GPT-6 Astra tasks on EHR inbox work as clinically safe. Take the claim at face value and do the two conversions the number invites.
# 1. What the interval actually permits. rated safe 99% of n = 325 # ~3 tasks not rated safe Wilson 95% interval [97.2%, 99.6%] unsafe rate, upper end 2.8 per 100 tasks # 2. What that rate meets in production. messages / clinician-day 14 # conservative public figure clinical days / year 230 messages / clinician-year 3,220 at 1.0% unsafe -> 32 unsafe dispositions per clinician-year at 2.8% unsafe -> 90 unsafe dispositions per clinician-year # 3. And the size of the evidence base, in the same unit. 325 tasks = about 10% of ONE clinician-year of messages
None of this says the number is wrong. It says the number is a per-task rate presented without its denominator, and clinical inboxes are a high-denominator workload — the regime where a 99% per-task safety rate and an unacceptable annual harm count are the same fact. It also says the entire public evidence base for the feature is about a tenth of one clinician's year, which is a reasonable size for a capability demonstration and far too small to characterise a tail.
Two further cautions on vendor-reported safety for this workflow. "Clinically safe" is not "correct" — a reply that is safe and wrong is still a quality failure and still generates a follow-up message, which lands back in the inbox you were trying to shrink. And a reviewer rating a task is not a patient experiencing an outcome; the rating has no way to see the message the patient did not send because the first reply looked authoritative.
Computer-use agents change the attribution picture, not just the capability. An agent that drives the EHR's interface is acting inside a human's authenticated session, so the access logs — the ones your compliance team will read after an incident — record the clinician, not the agent. If you deploy this shape, insist on a distinct service identity and a per-action marker in the audit trail before go-live, because you cannot retrofit attribution onto events already written. See audit trails and agent identity and permissions.
Step 4Who signs, and the three constraints that are genuinely binding.
The regulatory surface for inbox triage is narrower than teams expect and the parts that bind are specific:
- A clinical determination needs a licensed clinician, and triage urgency is one. The pattern that holds is the one from prior authorization: the agent may say "this needs a clinician" and may say "this is administrative", but the branch where it downgrades a clinical concern should not exist as code. Downgrades are an escalation to a human, always.
- Standing orders are the mechanism that makes the protocolled pool legal, and they are per-organisation and per-state. The agent does not create authority; it routes into authority that already exists on paper. Get the list of existing standing orders before designing the taxonomy, because that list is the taxonomy's third class.
- Disclosure is becoming the default expectation on the patient-facing side. A qualitative study published in 2026 found patients broadly comfortable with AI-drafted portal messages and frequently reading them as a reasonable response to clinician workload — comfort that the literature ties to being told. Treat the disclosure as load-bearing for that acceptance rather than as a legal checkbox; disclosure and content provenance covers the mechanics.
What is not binding, and gets over-engineered: the drafting step itself, when a clinician reviews and sends. That is the clinician's message, signed by the clinician, and the governance question it raises is about display and edit affordances rather than about model risk. Spend the compliance effort on the dispositions you automate.
Step 5Evaluate on your own adjudicated inbox, stratified, and score recall on one class.
No public benchmark will settle this for you. There are now several — HealthAdminBench on healthcare administration tasks for computer-use agents, EHR-Complex and HealthAgentBench on clinical agent reasoning, all posted to arXiv during 2026 — and they are worth reading for task design, but an inbox is a local distribution. Your panel's mix of chronic disease, your portal's message template, your organisation's standing orders and your patients' phrasing determine the answer, and none of them is in a benchmark.
Build the eval set the expensive way, once:
- Sample stratified by disposition, not at random. The urgent class is the one that matters and it is rare, so a random sample of 500 messages will contain too few to measure. Oversample it deliberately and reweight.
- Adjudicate with two clinicians and record disagreements rather than resolving them away. Inter-rater agreement on urgency is the ceiling on any score you can report, and if it is moderate you have learned something more important than the model's accuracy — see judge calibration.
- Report recall on "clinician, urgent" and precision everywhere else. One headline accuracy figure over five classes is dominated by the three administrative ones and will look excellent while the only dangerous error rate drifts. Set a ship-blocking floor on urgent recall and let the other classes be tuned for throughput.
- Grade the citation, not just the answer, on the auto-closed class. A reply that is correct from a record field the agent did not actually read is a future incident with a passing test.
Then keep it alive. Message mix shifts seasonally and after any portal change, and a triage model is exactly the kind of component whose inputs move without anyone filing a ticket — eval-set maintenance is the ongoing cost, and it is not optional for a workload this adversarial to staleness.
Step 6Roll out by disposition, and measure the thing the drafting studies could not find.
Sequence the launch by which destination absorbs a mistake, which means administrative classes first and clinical classes last — the opposite of the order vendors demo:
- Phase 1 — suggest only. The agent proposes a disposition; staff route. You get labelled data and a measured agreement rate before anything is automated, and you learn your misrouting baseline, which is usually the biggest single win available.
- Phase 2 — auto-route the non-clinical pool. A billing question sent to billing is wrong in a way that costs a forward, not a patient.
- Phase 3 — auto-close the record-lookup class, with the audit queue still on. Keep it on until the class has months of clean history, then sample rather than review.
- Phase 4 — route into the protocolled pool, strictly within existing standing orders.
- Drafting, whenever you like. It is the lowest-risk and lowest-value component. Ship it where the evidence supports it — messages a clinician was always going to answer — and do not let it run on anything the router closed.
The metric set is short and it deliberately excludes the one everybody reports. Clinician touches per 100 arriving messages is the number the whole build moves; turnaround time and minutes-per-draft are the numbers that went sideways in the published studies. Pair it with urgent-class recall against your adjudicated set, the auto-closed rate with its reversal rate, and — the one that tells you whether you shrank the work or moved it — follow-up messages per closed thread. A triage system that closes a thread and generates 1.4 new ones has not reduced volume; it has split it, and the drafting literature's missing 64 hours is probably hiding in exactly that number.
Start by measuring misrouting for two weeks with no model in the loop: what fraction of messages that reached a clinician needed a clinician at all. In most inboxes that single number justifies the project and sizes it, and it is the one the drafting-first architecture never collects. Then build the five-class router, set a ship-blocking floor on urgent recall, keep abstention wired to the urgent class, and treat the drafter as a later, optional stage that only runs where a human will reply anyway. Related reading: healthcare agents for the clinical-safety frame, ambient clinical documentation for the other half of the note-and-inbox burden, customer-support agents for the deflection maths in a domain where a wrong answer is cheaper, and human in the loop for keeping the audit queue a control rather than a formality.