Blast Radius

Probability and magnitude are two different budgets.

Risk is roughly probability times cost, and almost all the attention in agent engineering goes to the left-hand term. Better prompts, better tools, better evals, a human in the loop: each of these buys down the chance of a bad action. None of them changes what the bad action costs once it lands.

That asymmetry matters because the two terms have very different error bars. You can measure a success rate on a held-out set and be wrong about production by a few points. You can measure a blast radius exactly, today, by reading a set of grants — and the answer does not move when the model changes underneath you.

• Probability is estimated. It comes from evals, and it degrades quietly under distribution shift, prompt changes, and model swaps.
• Magnitude is enumerated. It comes from credentials, network reach, and tool definitions. You can write it down.
• Only one of them survives a surprise. A novel failure mode invalidates your probability estimate. It does not enlarge your blast radius, provided the radius was drawn with a mechanism rather than an instruction.

Four dimensions, and you have to name all four.

"Blast radius" is not one number. It is the product of four independent quantities, and shrinking three of them while leaving the fourth wide open buys almost nothing.

• Reach — what the agent can touch at all. Which hosts, which buckets, which tables, which repositories, which tenants. This is set by network egress and by credential scope, and it is usually the largest of the four.
• Authority — what it may do to what it touches. Read, write, delete, grant. The gap between read and delete on the same resource is often the entire difference between an embarrassing incident and an unrecoverable one.
• Rate — how much, how fast, before anything notices. An agent with delete authority over one row and an agent with delete authority over one row ten thousand times per minute have the same scope and wildly different radii.
• Reversibility — whether the action can be taken back, by whom, and within what window. A refundable charge, a soft-deleted record and a sent email are three different worlds.

The four compose multiplicatively, which is the useful part: capping the rate on a broad grant can be cheaper and faster than narrowing the grant, and a reliable undo can be worth more than either. See autonomy levels for the dial these four sit behind.

The radius is drawn by credentials, and credentials outlive their reasons.

In practice a running agent's blast radius is whatever its tokens permit, and tokens are handed out for a phase of work and then never taken back. The agent needed write access to the production database while somebody was building the integration; the build finished; the grant did not. Nothing in the system marks the difference between an access that is still justified and one that is merely still valid.

Two habits make this worse than it looks:

• Impersonation by default. An agent that acts with a human's credentials inherits that human's entire authority, not the slice the task needed. A support agent running as a support engineer can do everything a support engineer can do, forever. This is ambient authority, and it is the default shape of most integrations.
• Scope granularity that does not match tasks. Real permission systems offer scopes like repo or db.write. Tasks want "this repository, this branch, for the next twenty minutes". When the available scope is coarser than the task, teams grant the coarse thing, and the radius is set by the vocabulary of the permission system rather than by anyone's intent.

The consequence is the one that surprises people: no adversary is required. An agent doing exactly what it was asked, with access nobody remembered it had, is sufficient to produce the largest incidents in this class. See agent identity and permissions.

Bound it with a mechanism, then measure it as a question.

A blast radius drawn by an instruction is not drawn. "Do not delete anything in production" is a probability control wearing a magnitude control's clothes — it lives in the same text the model is free to reason around, and it is invisible to everything else in the system. A radius has to be enforced somewhere the agent cannot argue with: the credential, the network, the tool implementation, the database role.

• Give the task the credential, not the agent. Short-lived, minted per run, scoped to the resources that run named. Expiry does more work here than scope does, because expiry needs no foresight.
• Cap rate and volume at the tool. A tool that refuses its eleventh destructive call in a run turns an unbounded incident into a bounded one, and does it without predicting the failure.
• Buy reversibility before you buy autonomy. Soft deletes, staged writes, a queued send. Where undo exists, the radius shrinks by a factor you can actually feel; see human in the loop for where a person belongs in that path.
• Separate the read grant from the write grant. Most agent value is in reading widely. Most agent damage is in writing widely. They almost never need to be the same credential.