Physical Actuation: Safety Without Undo

Name the four assumptions that stop holding.

It is worth being precise about what changes, because "physical is riskier" is not actionable and leads teams to bolt an extra approval prompt onto a loop whose real problem is elsewhere.

• Reversibility is gone. A dispensed reagent is consumed, a cut is cut, a mixed batch is mixed. Whole families of control — dry-run first, snapshot and restore, compensating transaction — either do not exist or exist only as a rehearsal in a different medium. Everything the undo and reversibility playbook offers a software agent has to be re-earned here.
• Idempotency is gone. "Retry on timeout" is the single most common resilience pattern in agent infrastructure and it is actively dangerous against an actuator: the command may have executed and the acknowledgement been lost, so the retry moves the axis twice. Compare idempotency and retries — the same discipline, with the failure mode upgraded from a duplicate charge to a collision.
• The sandbox is gone. There is no container for a centrifuge. Sandboxing works because you can give the agent a fake world that behaves like the real one; simulation is genuinely useful here (Step 4) but it is a model of the physics, not the physics.
• Observation lags and is partial. A software agent reads exact state. A physical one reads sensors that are sampled, noisy, occasionally wrong, and silent about the thing that actually matters — the tube that was in the wrong rack. Plan for the agent to be confidently mistaken about the world.

Put the limits in the driver, and do not confuse a driver limit with a safety function.

The design that has emerged — and that Anthropic's Model Hardware Standard research preview makes explicit — is that a device should describe itself and enforce its own limits, so the envelope holds regardless of which model is driving. That is the right architecture. The device knows its travel range, its maximum payload, its temperature ceiling; encoding those in the driver means an agent that has been talked into a bad plan still cannot execute the part that exceeds them, and it means the same limits apply to a different model, a different orchestrator, or a script written by a human in a hurry.

What it is not is a safety function in the machinery-safety sense, and running these together in your head is how a programme ends up with an impressive-looking control that stops nothing. The two layers do different jobs:

• A driver-enforced limit is ordinary software in the command path. It is excellent at preventing a bad instruction from becoming a bad motion, and it is subject to every failure ordinary software has: it crashes, it has bugs, it can be bypassed by whatever talks to the hardware underneath it, and it is only as good as the numbers somebody typed into the device description.
• A rated protective function — an interlock, a light curtain, a safety-rated speed-and-separation monitor, an emergency stop circuit — is assessed against functional-safety requirements, is typically required to reach a performance level with a redundant architecture, and is designed to act when the control system is the thing that has failed. The 2025 revision of ISO 10218 (parts 1 and 2, replacing the 2011 editions and absorbing the collaborative-application content formerly in ISO/TS 15066) made these functional-safety requirements more explicit and added cybersecurity requirements alongside them.

Write both into the design, label which is which, and never let a document claim the model "cannot" exceed a limit when the only thing preventing it is code the model's own toolchain loads.

Authorise the envelope, not the step.

Human-in-the-loop as usually practised — surface the proposed action, wait for a click — does not survive contact with a machine that runs a 40-step protocol at machine tempo. You cannot put a person between a command and a servo, and a team that tries ends up either with an operator who approves everything reflexively or with a run that takes longer than doing it by hand.

So move the decision earlier and make it bigger. Before a run starts, a named human authorises a bounded envelope: which devices, which consumables, which parameter ranges, which durations, what the agent may do on its own initiative when something is out of spec, and what it must stop for. Inside the envelope the agent runs unattended; the edge of the envelope is a hard stop, not a prompt. This is the same structure as human-in-the-loop and autonomy levels, with the granularity moved from action to session because latency forced it.

Rehearse in simulation, then rehearse on cheap material.

You still need a staging environment; it is simply built differently (staging environments for agents). Three tiers, and skipping the middle one is the common mistake:

• Simulated devices. Drivers that accept the full command surface and return plausible telemetry. This catches the overwhelming majority of agent errors — wrong units, wrong axis, a step issued before the previous one completed, a plan that assumes a tool it does not have — at zero material cost, and it is where your regression suite should live.
• Real hardware, worthless material. Water instead of reagent, a dummy plate, a foam block instead of the part. This is the tier that finds what simulation cannot: timing that only appears under real motion, a gripper that slips, a lid that was not where the model said it was, the cable that fouls at full extension.
• Production, attended, then production unattended. Attended runs on real material are the acceptance test, and the promotion criterion should be a count of clean runs, not a demo.

Simulation earns its keep and also lies in a specific direction: it is built from the same assumptions the agent has, so it validates the plan and not the world. Anything about variance in the physical inputs — a lot that behaves differently, a component out of tolerance, an operator who loaded the rack backwards — is invisible until tier two.

Define the safe state per device, and keep stop authority out of the software path.

"Kill switch" is one word for several different mechanisms here, and the useful discipline is to write down, for each device, what "stopped" actually means. Cutting power to a robot arm mid-motion may drop the payload. Cutting power to a heater may be exactly right, or may ruin an eight-hour run that could have been paused. A pump stopped mid-transfer leaves fluid somewhere. There is no generic safe state; there is a per-device answer, and somebody has to decide it before the run, not during the incident.

Three requirements that are easy to state and frequently missing:

• The physical stop does not depend on the agent stack. Emergency stop is a hardware circuit. If the only way to halt the cell is an API call that goes through the orchestrator that is currently wedged, you do not have a stop — you have a request. The kill-switch discipline applies, with the addition that the last resort must be a button on a wall.
• Loss of contact means stop, not continue. Give every device a watchdog: no heartbeat from the controller within n seconds and it moves to its defined safe state on its own. A queued command list that keeps executing after the agent has crashed is the worst of both worlds — autonomous action with nobody watching.
• Resumption is a decision, not a default. After any stop, the physical state is unknown until observed. Resuming from the step you think you were on is how a run that was recoverable becomes a crash. Re-establish state from sensors, or require a human to confirm it, then continue (graceful degradation and fallback).

Measure stops and envelope exits, not task success.

The dashboard that gets built first — protocols completed, hours saved, throughput versus the manual process — is the one that tells you least about whether the programme is safe to expand. The numbers that matter are the ones nobody asks for:

• Unplanned stops per run, split by cause: envelope violation caught, watchdog fired, human intervened, device fault. A falling rate is the only honest evidence that autonomy should widen.
• Runs that ended outside the envelope, including the ones that produced a fine result anyway. A good outcome from an unauthorised action is a near miss, and near misses are the only cheap data you get.
• Time to physical stop, measured, not assumed — from the decision to halt to the machine actually being still, including whoever had to walk to the room.
• Material and time consumed by failed runs, which is the real unit economics of the deployment and the number a scale-up business case is usually missing.

Record-keeping deserves the same seriousness. A physical run leaves evidence in the world that outlives your trace retention, so the run record — envelope, authoriser, device firmware and driver versions, full command log, sensor stream, and what the agent was told — is the artefact an investigation will need (audit trails, incident response for agents). And be clear who the operator of record is: machinery and workplace-safety obligations attach to a named person and an organisation, never to a model, so accountability and roles is a prerequisite here rather than a follow-up.

Related: the deployment safety checklist for the pre-launch gate this extends, repairing agent side effects for the cleanup discipline when the effect is not repairable by a script, and insurance and liability for who carries the cost when it goes wrong.