Dry Runs & Preview Modes

Step 1A preview is a claim; the question is who makes it.

Show a user "I'll archive the 14 threads older than 90 days and leave the starred ones" and they have been given a claim about a future action. There are only two things that can have authored that claim. Either the model wrote it, by describing the tool call it is about to make — in which case it is a summary, subject to every ordinary generation error, and nothing in the system has checked it against the tool. Or the tool wrote it, by being invoked in a mode that computes the effect and returns it without committing — in which case it is a measurement.

Almost every agent UI today ships the first kind and presents it with the visual grammar of the second. The failure is not that the model lies; it is that the model has no access to the information. It does not know that the archive endpoint treats "older than" as strictly greater, that 2 of the 14 were already archived, or that a shared label will cascade. Its sentence is a reasonable paraphrase of its intent, which is exactly the thing the user already understood and the opposite of what they needed to see.

Step 2Two previews, one of which is produced by the real path.

Infrastructure tooling worked this out two decades before agents, and the distinction it landed on is the one to copy. kubectl apply --dry-run=client renders the object your command would send and prints it; it never leaves your laptop, so it cannot know what the cluster would do with it. kubectl apply --dry-run=server submits the request to the API server, which runs the full admission chain — defaulting, mutating and validating webhooks, quota checks — and then declines to persist. Same flag, same verb, and a gulf between them: the client-side preview is a rendering of your input, the server-side preview is the actual decision with the write suppressed at the last step.

Terraform draws the same line with a file. A bare terraform apply computes a fresh plan, shows it, and asks; between the showing and the asking the world can move. terraform plan -out=tfplan followed by terraform apply tfplan applies that plan — the approval is attached to an artifact, not to a remembered screenful of text. SQL's EXPLAIN sits at the weaker end on purpose: it returns the planner's intention, and EXPLAIN ANALYZE exists precisely because the intention and the execution are known to differ.

Stripe's test mode is the instructive non-example. It is a complete, faithful, separately-ledgered world, and a successful test charge tells you almost nothing about the live one, because the thing you were worried about — this customer, this card, this balance — is not in it. A sandboxed rehearsal is a development tool. It is not a preview of the action you are about to take.

Step 3Build it as a parameter, never as a sibling function.

The implementation rule follows directly: dry run is an argument to the one code path, threaded down to the last call, where the commit is skipped and the computed effect is returned instead. The moment you write a second function — preview_archive() next to archive() — you have two implementations of one policy, and they will drift on the first edge case anybody fixes in a hurry. Nothing in your test suite notices, because both functions still pass their own tests.

# one path, one policy, two endings
def archive_threads(query, *, commit: bool):
    targets = resolve(query)              # the real selection logic
    effect  = plan_effect(targets)        # 12 archive, 2 no-op, 1 label cascade
    if not commit:
        return effect                    # this object IS the preview
    return apply(effect)                 # applies the object shown, not a re-plan

# the approval binds to the effect, not to the request
effect = archive_threads(q, commit=False)
token  = sign(effect)                    # hash of the exact planned effect
# ... human approves, minutes later ...
apply_approved(token)                   # re-plans; MISMATCH -> stop, re-ask

The last three lines are the part people skip. An approval granted against a preview is only valid while the preview is still true, and between the click and the call a mailbox gets new mail and a branch gets new commits — the classic time-of-check-to-time-of-use gap. Signing the computed effect and re-checking it at commit time turns a stale approval into a second prompt rather than a surprise. Terraform's plan file is this idea with a filename.

Step 4Some actions cannot be previewed — price those honestly.

A dry run requires that the effect be computable without the effect occurring, and plenty of real actions fail that test: sending an email, charging a card, calling a third-party API whose response depends on state you cannot read, anything whose outcome is decided by a system that will not tell you in advance. Faking a preview for these is worse than offering none, because a confident preview of an unpreviewable action is a false measurement wearing the costume of a real one.

For those, buy safety somewhere else on the budget: narrow the task scope so the worst case is small, make the action reversible or at least repairable, add an idempotency key so a retry is not a second charge, and keep enough of a record that repair is a procedure rather than an investigation. Reversibility and preview are substitutes in the same budget — cheap undo buys you the right to skip a preview, and an unpreviewable, irreversible action is the one case that genuinely needs a human to originate it rather than approve it.