The rate was per task, the inbox is per day

OpenAI put computer-use agents into hospital software on 16 November 2026 and reported reviewers rating 99% of 325 EHR-inbox tasks clinically safe. Give that rate a denominator — about 3,220 messages per clinician-year — and it describes 32 to 90 unsafe dispositions a year, from an evidence base the size of one clinician-month. The value claim has the same missing denominator: a cohort study of 983,154 portal replies found an AI drafter 29.5% faster on the messages it touched and 8.6% slower on the queue as a whole, because a draft creates review work on messages nobody needed to read.

By Agentic AI Wiki 11 min read

OpenAI put computer-use agents into hospital software on 16 November 2026 and reported that reviewers rated 99% of 325 EHR-inbox tasks as clinically safe. Convert that per-task rate into the unit a clinician actually lives in — thousands of messages a year — and the same number describes somewhere between 32 and 90 unsafe dispositions per clinician-year. The denominator is the whole story, and it is missing from both the safety claim and the value claim for this entire product category.

What was announced

At HLTH 2026, OpenAI expanded ChatGPT for Healthcare with computer-use agents — models that read the screen, click, type and move between applications rather than calling an API — pointed at administrative clinical work.

ElementAs reported from the announcement
Date16 November 2026, at HLTH 2026
AgentsComputer-use agents reported as powered by GPT-6 Astra and GPT-6 Luna
Named use casesPrior authorizations, referrals, discharge planning, EHR inbox work
PluginsMore than 20, including Availity, Elation Health, Redox, TriZetto, UpToDate Expert AI, Luma Health
Safety figureReviewers rated 99% of 325 GPT-6 Astra tasks on EHR inbox work as clinically safe — OpenAI's own evaluation
AvailabilityEnterprise workspaces, with admins enabling plugins; selected plugins via a free ChatGPT for Clinicians account requiring US verification

A provenance note. All of the above is from trade-press and aggregator coverage on announcement day rather than from OpenAI's own post, and the plugin count is reported variously as 20-plus or 21. The 99%-of-325 figure is consistent across the coverage and is explicitly a vendor self-evaluation. The arithmetic below uses only that figure and public volume data; none of it depends on the parts that vary.

Ninety-nine percent of 325 is a capability demo, not a safety case

Start with the interval. Three failures out of 325 is a point estimate of 99%, and a Wilson 95% confidence interval around it runs roughly 97.2% to 99.6%. So the evidence is consistent with an unsafe rate anywhere from 0.4 to 2.8 per hundred tasks — a factor of seven, which is what n = 325 buys you at this success rate.

Now give it a denominator. One large health system published a figure of fewer than 14 In Basket messages per physician per day, describing it as among the lowest in the Epic community; take it as a deliberately conservative floor across 230 clinical days:

messages / clinician-year   14 x 230  =  3,220

at 1.0% unsafe   ->   32 unsafe dispositions / clinician-year
at 2.8% unsafe   ->   90 unsafe dispositions / clinician-year

eval set size         325 tasks  =  about one clinician-MONTH
Evaluation size against inbox workload One clinician handles about 3,220 inbox messages a year at a conservative 14 messages per clinical day across 230 days. The published vendor evaluation covered 325 tasks, which is roughly one clinician-month of about 268 messages. The evidence base is about a tenth of a single clinician-year. Inbox messages, same unit on every bar One clinician, one year 3,220 Published vendor eval set 325 One clinician, one month 268 0 1,000 2,000 3,000 Clinician-year uses 14 messages per clinical day across 230 days — a published figure described as among the lowest in its EHR community, so it is a floor rather than a mean. At a 1.0% unsafe rate that year holds 32 unsafe dispositions; at 2.8%, ninety.
The published evidence base is roughly one clinician-month of traffic. The workload is per year, per clinician, across a workforce.

None of this says the number is wrong or the product is bad. It says a per-task rate is not a safety case for a high-denominator workload, and clinical inboxes are the high-denominator workload par excellence. The regime where "99% safe" and "unacceptable annual harm count" are the same sentence is exactly this one, and it is why the useful question at procurement is never the headline percentage but what rate, times what volume, lands in front of which reviewer.

Two further cautions specific to this claim. "Clinically safe" is not "correct" — a reply that is safe and wrong still generates a follow-up message, which lands back in the inbox the agent was bought to shrink. And a reviewer scoring a task cannot see the message a patient did not send because the first answer sounded authoritative; the rating has no access to the outcome. The general form of this problem is in human baselines in agent evals, and the sample-size half is eval variance and statistical power.

The value claim has the same missing denominator, and here the evidence is unusually good

This is the rare clinical AI workflow with real deployment data, because draft-reply tools shipped early and at scale inside Epic and got studied on production traffic. The results are not what the category's marketing implies.

Draft-tool effect on message turnaround, measured two ways In a cohort study of 983,154 patient-portal message responses from 1,542 clinicians, adoption of an AI draft-response tool was associated with message turnaround time 8.6 percent slower at clinic level, while actually using a draft on a given message was associated with 29.5 percent faster turnaround. The two measurements point in opposite directions. Change in message turnaround time Clinic level — tool adopted by the team +8.6% slower Message level — a draft actually used 29.5% faster -30% -20% -10% 0 +10% Cohort study: 983,154 portal message responses from 1,542 clinicians. A draft speeds up the messages a clinician was always going to write, and adds a review step to the ones they were not. After-hours work did not meaningfully fall.
The same tool, measured two ways: faster on the messages it touches, slower on the queue as a whole.

A cohort study in JAMA Network Open covering 983,154 patient-portal message responses from 1,542 clinicians found that adoption of an AI draft-response tool was associated with an 8.6% increase in message turnaround time, while actually using a draft on a given message was associated with a 29.5% reduction. After-hours portal work did not meaningfully fall. An earlier quality-improvement study of 162 clinicians found significant improvement on burden and burnout measures with no change in time spent, and a pediatric study reported perceived savings of roughly 1.2 minutes per message with burnout unchanged.

Multiply that perceived 1.2 minutes by the same 3,220 messages and you get about 64 hours a clinician-year. The time studies do not find 64 hours. They find no change — and at clinic level, a queue moving 8.6% slower once the tool is available.

One explanation fits all four results: a draft invites a clinical review of messages that would otherwise have been dispatched in seconds, or never reached the clinician at all. The tool speeds up the hard messages and converts the easy ones into reviewable artefacts. Net of both, the step that got automated was the cheap one. That is not a model-quality problem and a better model does not fix it, because the mechanism is the review the draft creates.

So the two numbers in this story fail identically. The safety figure is a per-message rate presented without the volume that makes it matter; the value figure is a per-message saving presented without the messages it caused someone to look at. Both are true. Neither is the quantity a buyer needs.

Computer use changes the audit trail, not only the capability

There is a second-order consequence of the computer-use shape that is easy to miss in a capability announcement. An agent that drives the EHR's own interface is operating inside a human's authenticated session. It has no identity at the application layer, because the application cannot tell it apart from the clinician whose session it is using.

That means the access logs — the ones a compliance team reads after an incident, the ones that carry the regulatory weight in healthcare — record the clinician. Not the agent, not the model version, not the plugin. Every downstream process that depends on attribution inherits the ambiguity: incident reconstruction, break-the-glass review, a patient's request for an account of who saw their record.

It is also not retrofittable. You cannot add attribution to events already written, so the only moment to insist on a distinct service identity and a per-action marker is before go-live. This is the confused-deputy shape, arriving through the GUI rather than through a token, and the mitigations are the ordinary ones: a separate identity, a marker in the audit trail, and a scope that is narrower than the human's. See audit trails and third-party agents logging in as your users.

The same point has a regulatory edge now. Colorado's replacement AI statute, due January 2027, requires an explanation of an adverse outcome within 30 days and a path to meaningful human review; the EU's Article 50 transparency duties have applied since 2 August 2026. Both are per-decision artefacts, and an action attributed to the wrong actor cannot produce either. The duties that landed on schedule in 2026 were exactly the artefact duties, which makes attribution an engineering dependency rather than a compliance nicety.

What to build instead

The inversion in the draft-tool data points at a different architecture, and it is the one nobody demos because it has no visible output.

  • Make the primary output a disposition, not prose. Five destinations cover inbox traffic: closed from the record, non-clinical pool, protocolled clinical pool, clinician routine, clinician urgent. Moving a message from "clinician routine" to "non-clinical pool" removes a clinician touch entirely rather than shortening it, and three of the five destinations need nobody licensed.
  • Measure clinician touches per 100 arriving messages. Turnaround time and minutes-per-draft are the metrics that went sideways in the published studies. Touches is the one the build actually moves.
  • Score recall on one class. A single accuracy figure across five classes is dominated by the three administrative ones and will look excellent while the only dangerous error rate — missed urgency — drifts. Put a ship-blocking floor on urgent recall and tune the rest for throughput.
  • Wire abstention to the urgent class. "I cannot tell" costing a clinician thirty seconds is the cheapest control in the system.
  • Watch follow-ups per closed thread. A system that closes a thread and generates 1.4 new ones has split the work, not reduced it — and that is the most likely hiding place for the drafting literature's missing 64 hours.
  • Treat the drafter as a later, optional stage that runs only where a human was always going to reply. It is the lowest-risk and lowest-value component, and letting it run on messages the router already closed is how you recreate the 8.6%.

The full build order, the standing-order constraint that makes the protocolled pool legal, and how to construct a stratified eval from your own adjudicated inbox are in clinical inbox triage agents.

FAQ

Is a 99% clinical safety rate good enough to deploy an inbox agent?

It depends entirely on volume, and the rate alone cannot tell you. At a conservative 3,220 messages per clinician-year, a 1% unsafe rate is about 32 unsafe dispositions a clinician-year, and the upper end of the 95% interval around 99%-of-325 permits about 90. Ask what rate times what volume reaches which reviewer, then decide which dispositions you are willing to automate.

What did the largest study of AI draft replies actually find?

A JAMA Network Open cohort study of 983,154 responses from 1,542 clinicians found adoption associated with an 8.6% increase in turnaround time and actual use of a draft associated with a 29.5% reduction, with no meaningful fall in after-hours portal work. The most consistent finding across the literature is improved perceived burden with unchanged time.

Why would an AI drafting tool make message turnaround slower?

Because a draft creates a reviewable artefact on messages that would otherwise have been dispatched in seconds or routed away from the clinician entirely. The tool speeds up the messages a clinician was always going to write and adds a review step to the ones they were not. A better model does not fix this; a router that decides who touches the message does.

Who shows up in the audit log when a computer-use agent works in an EHR?

The clinician whose authenticated session the agent is driving. A GUI-level agent has no application-layer identity, so attribution has to be added deliberately — a distinct service identity and a per-action marker — before go-live, because you cannot retrofit it onto events already written.

Is patient-portal message volume still growing?

Yes. National Epic data shows patient-authored portal messages rose 153% between 2020 and 2025, from about 2.2 to 5.4 per sending patient per year, while clinician and staff replies rose 24% to 5.70 per patient per year after peaking at 6.86 during the pandemic. The reply side is near its ceiling; the arriving side is not.

Further reading

On this wiki:

Sources:

  • JAMA Network Open — the draft-reply cohort and quality-improvement studies.
  • OpenAI — the ChatGPT for Healthcare announcement.