Notifications & Digests

Two policies, and only one of them is about the task.

An unattended agent produces two outputs on every run: what it did, and what it told you. Teams tune the first and let the second fall out of whatever the framework does by default — usually "message on completion" — and then wonder why nobody reads the messages.

• The task policy asks what to do. The notification policy asks whether a human needs to know now. These have different inputs: the second one depends on who the recipient is, what they can act on, what time it is for them, and what they were already told this morning — none of which the task policy has any business knowing.
• Separating them makes both testable. You can evaluate the notification policy on logged runs without re-running the agent: replay a week of outcomes, ask what would have been sent, and grade it. That is a cheap eval loop the coupled design does not permit.
• Completion is not an event. "Task finished successfully" is the most-sent and least-valuable notification in the category. Success is the expected case; if it needs announcing, the run belongs in a digest, not in a push.
• The default must be silence. Every message needs a reason it could not wait. Inverting the default — notify unless suppressed — is how a well-behaved agent becomes a firehose after its third feature.

The cost is asymmetric, non-stationary, and paid by someone else.

Precision and recall trade off in every classifier, and here the exchange rate is brutally lopsided in a way that is easy to miss because the loss lands outside your metrics.

• A false positive is not one bad message. It is a small, permanent reduction in the attention paid to every future message on that channel. The damage compounds; the benefit of the true positive it was traded against does not.
• Mute is terminal. Once someone turns off a notification category they effectively never turn it back on, and your recall for that recipient goes to zero — including for the one alert the whole system exists to deliver. Treat the mute rate as the guard metric on everything else.
• Habituation is invisible in your logs. Delivery succeeds, the message is technically opened, and nothing happens. Open rates flatter a channel that has already stopped working; only action tells you anything.
• The person who bears the cost is not the person who tunes the threshold. Builders see one agent; a recipient sees the sum of eleven. Budget at the recipient level, per Step 5, or every individually-reasonable agent will collectively be unusable.
• Under-notifying has a real cost too, and it is quieter. An agent that silently abandons a task, or waits on an approval nobody knew was pending, fails in a way that produces no signal at all. That is the case for a dead-man's switch rather than for a lower threshold.

Route by decision deadline, not by importance.

"How important is this?" is unanswerable and every agent answers it optimistically. "When does this stop being actionable?" is a question with a checkable answer, and it maps cleanly onto channels.

• Interrupt now — the window closes in minutes or hours, and a human decision changes the outcome. A blocked approval on a deadline, a spend anomaly still accruing, an irreversible action awaiting confirmation.
• Digest — actionable, but tomorrow morning is fine. Completed work, recommendations, drift that is not yet urgent. Batching is not a downgrade; it is the correct channel for most of what an agent produces.
• Log only — no decision is implied. Retrievable when someone asks, never pushed.
• Escalate on non-response, do not repeat. If nobody acts before the window closes, the next step is a different recipient or a documented safe default — not the same message again. Re-sending trains people that the first message was ignorable.
• Respect the recipient's clock, and define the exception in advance. Quiet hours with a documented break-glass class is the honest design. An agent that decides at 03:00 that this one is important enough has just re-derived the mute button.

Write the notification so the decision fits inside it.

A notification that requires opening a dashboard to understand has failed, because the interruption cost was paid and the decision was not made. The whole point of the format is to close the loop in the notification itself.

• Lead with what changed and what is being asked. First line: the decision or the fact. Not the agent's name, not the run ID, not "I have completed my analysis of".
• Include the evidence that would change the answer. The number, the threshold it crossed, and the window it was measured over. One line of grounding turns a claim into something a person can accept or reject without leaving the message — the discipline in source-attribution UX.
• State what happens if nobody replies. "Will proceed at 14:00" and "will abandon at 14:00" are different messages and demand different urgency. Silence about the default is the most common defect in agent notifications.
• Put the reversal next to the action. Approve and undo belong in the same surface, per undo and reversibility; a one-tap approval with no visible way back raises the stakes of every notification and slows all of them down.
• Calibrate the language to the confidence. An agent that phrases a 60%-confidence finding the same way as a certainty is training the reader to discount both. Hedged language is a feature here — see designing for trust.
• Digests are ranked, not chronological. A digest sorted by time is a log. Lead with the items that need a decision, put the completed work below, and make the count of the second group visible without listing it.

Rate-limit outside the agent's judgement.

Every argument for sending one more message is made by the component that wants to send it. The budget therefore cannot live in the prompt, or in the agent's reasoning, or in a per-agent config that each team tunes for its own agent.

• Budget per recipient per period, across all agents. A shared quota in the notification service, not per agent. When it is exhausted, further messages downgrade to the digest — the agent does not get a vote, and it does not get to declare an exception.
• Deduplicate on a stable key. Same condition, same object, same open incident is one notification with an updated state, not eleven. Most notification storms are one condition observed repeatedly.
• Suppress on correlation. When forty line items breach at once, something upstream broke; send one message about the upstream cause and a count. Fan-out during an incident is how the channel gets muted on the worst possible day.
• Reserve a break-glass class and keep it small. A tiny set of conditions bypasses the budget — irreversible action pending, safety condition, spend still accruing. If more than a handful of conditions qualify, none of them do.
• Put a dead-man's switch on silence. An agent that should report daily and has not is a failure that produces no notification by construction. Alert on the absence, from outside the agent.
• Give the recipient granular control, and treat its use as data. Per-category rather than per-agent muting keeps the useful classes alive. When someone mutes a category, that is your precision metric telling you something the dashboard did not.

Measure action, not delivery.

Notification systems are usually instrumented on the sending side, which measures whether the code worked rather than whether the design did. Three numbers describe the health of the channel, and one of them is the number nobody collects.

• Actioned-within-window rate — of the messages that asked for a decision, what fraction got one before the deadline. This is the primary metric and it should be embarrassingly high; if it is not, you are sending things that did not need sending.
• Mute and unsubscribe rate — the guard metric. It moves slowly, never recovers, and it is the leading indicator that precision has slipped.
• Missed-signal rate — how often something happened that should have produced a notification and did not. Only obtainable by sampling incidents backwards and asking whether the channel carried them, which is why almost nobody has it and why it is the one worth building.
• Grade it on replay before you tune it live. Take a fortnight of runs, generate what the current policy would send, and have the actual recipients mark each one useful, tolerable or noise. It costs an afternoon and it is the only honest calibration available before shipping.
• Watch precision per category, not in aggregate. One noisy class hides inside a healthy average and takes the whole channel down with it when a recipient mutes at the channel level.

Related: interruption, steering and handoff for the in-session counterpart, approval and confirmation UX for the gate a notification often carries, cost and quota UX for another budget the user sees, and scheduled and triggered agents for the unattended runs that produce most of this traffic.