Infrastructure-as-Code Agents

9 min read

U20
Playbook · Coding & Computer-Use Agents

Infrastructure-as-code agents.

Every other coding agent has to build its own oracle before it can be trusted; this one is handed a good one for free, because terraform plan tells you what would happen to real infrastructure without touching it. That inverts the design: the agent's deliverable is not code, it is a plan whose every line falls into a class you have already agreed to auto-apply — and the entire engineering problem is that the plan is silent about precisely the four things that cause the outages.

STEP 1

The oracle already exists. Design around it, not around the code.

A test-generation agent has to invent a specification. A debugging agent has to build a reproduction before it knows anything. An infrastructure agent gets something neither can buy: a dry run that queries the live provider API, diffs it against declared state, and prints the exact set of creates, updates, replacements and destroys that an apply would perform. Terraform calls it plan; Pulumi calls it preview; the same primitive is cdk diff, kubectl diff, helm diff, and az deployment what-if.

It is cheap, read-only, and repeatable, which makes it the loop rather than the check at the end. So write the task down as a plan assertion and not as an instruction:

Task: give the ingest service read access to the events bucket.

Done when `terraform plan` shows:
  - exactly 1 resource to add (aws_iam_role_policy)
  - 0 to change, 0 to destroy
  - no diff outside module.ingest

Written that way, "did the agent succeed" is a parse, not a judgement. Written the other way — "add read access to the bucket" — you are back to reviewing code and hoping, which is the failure mode spec-driven development exists to name.

Corollary worth stating early: the agent should be able to run plan as many times as it wants, and apply never. Plan is a read; give it the credentials to do that freely and let it iterate. Apply is the side effect, and it belongs to the pipeline that runs after a human or a policy engine has classified the plan — not to the process that wrote it.

STEP 2

What the plan does not tell you — the four gaps that produce the incidents.

A plan is a partial oracle, and its blind spots are not randomly distributed. They cluster exactly where the expensive mistakes are.

  • (known after apply). Any value derived from a resource that does not exist yet is unknown at plan time, and it propagates: a security group id that is unknown makes every rule referencing it unknown too. The plan will confidently show you a change whose actual content it cannot compute. Worse, unknown values cannot be expanded, so a for_each over them fails or defers — meaning the number of resources in the real apply is not always the number in the plan.
  • Server-side behaviour. The provider sends a request; the cloud decides what to do with it. Defaults get filled in, values get normalised, some changes are applied as a replace even when the plan showed an update, and quota and eventual-consistency failures happen only on apply. The plan models the provider's intent, not the API's response.
  • Destruction, announced in the same tone as everything else. -/+ destroy and then create replacement is one line of output that can mean a new tag, or an empty database. The plan is accurate here and completely fails to be alarming, which for a human reviewer at 200 lines of diff is the same as being silent.
  • Everything outside the state file. Resources created by hand, by another stack, or by a controller reconciling in the background are invisible; drift is only caught to the extent the refresh caught it, and a plan run against a stale refresh is a plan against a fiction. Cross-stack ordering — this stack's plan is clean, and it breaks the one applied after it — is outside the frame entirely.

None of this argues against using the plan. It argues that the plan must be classified rather than read, which is the next step and the core of the playbook.

STEP 3

Classify the plan mechanically. Humans review the class, not the diff.

Emit the plan as JSON — terraform show -json tfplan gives you every resource change as structured data with its action, address, type and before/after attributes — and run a policy over it. This is the one place in an agent pipeline where a deterministic rule engine beats a model outright, and it is exactly the policy-as-code pattern applied to a diff:

  • Class A — auto-apply. Creates and in-place updates on stateless resource types, no deletes, no replacements, no IAM, no changes outside the module the task named, and no (known after apply) on any attribute the policy cares about.
  • Class B — human approval, normal review. Any in-place update on a stateful type, any change to networking reachability, anything that touches a resource the plan says is shared.
  • Class C — human approval, named reviewer. Every delete and every replace of a data-bearing type (databases, volumes, buckets, DNS zones, certificates), and every change to IAM, security groups, public exposure or encryption settings. These get their own reviewer list because they are the two categories from which unrecoverable outcomes come.
  • Class D — reject, do not show a human. The plan errored, the plan is empty when the task required a change, or the diff includes resources the task never mentioned. Bounce it back to the agent.

The value is not the taxonomy, it is the shift in what a reviewer is asked to do. "Read 300 lines of HCL diff and tell me if it is safe" is a task humans are measurably bad at and get worse at with volume — the precision-over-recall argument from code review agents, arriving here as review fatigue. "This plan is Class C because it replaces aws_db_instance.primary; approve or reject" is a task humans are good at.

Write the classifier before you write the agent. It is a day of work, it is useful on human-authored pull requests from the moment it exists, and it is the artefact that decides whether this programme can ever run unattended. An agent without it is a machine that generates plans for a queue of tired reviewers.

STEP 4

Give it schemas and state, because its memory of your provider is a year old.

The most common low-grade failure is not dangerous, just expensive: the agent writes an argument that does not exist, or uses one that was renamed two provider versions ago, and burns four plan cycles discovering it. Providers ship breaking changes on their own schedule and the model's weights do not.

  • Dump the schema and put it in the loop. terraform providers schema -json emits every resource type, attribute, and whether it forces replacement. Retrieving the relevant slice of that beats any amount of prompting about "use the correct arguments", and the forces replacement flag is directly the input the agent needs to avoid writing a Class C change by accident.
  • Give it the module catalogue, not the whole repo. Your organisation has approved modules with opinionated defaults; an agent that has not been shown them will write raw resources that pass plan and fail review. This is the same reuse problem as design-to-code, and the same fix: hand over the typed interface before the task.
  • Let it read state, carefully. The agent needs to know what exists. State files contain secrets in plaintext — that is a documented property of Terraform, not a misconfiguration — so read through terraform state list and targeted show calls rather than handing over the state blob, and treat plan JSON as sensitive for the same reason. Posting a full plan into a public pull-request comment is a credential leak waiting for its first repeat.

Run the whole thing with credentials scoped to plan-time reads, in the isolation the sandboxing and safe execution page describes. An IaC agent holds cloud credentials by construction; that makes it the highest-value prompt-injection target in your engineering org, and a third-party module's README is attacker-reachable text.

STEP 5

Three edits the agent must never make, and one decision it must never take.

The gaps in Step 2 are tolerable because the plan is otherwise honest. There are exactly three ways to make it dishonest, and they are all one-line changes that look like progress and make a red plan go green:

  • lifecycle { ignore_changes = [...] } — tells Terraform to stop reporting drift on an attribute. The diff disappears; the divergence does not. An agent that has been asked to "make the plan clean" will find this.
  • terraform state rm — removes a resource from state, so the plan stops mentioning it. The resource keeps running and keeps billing, now unmanaged and invisible.
  • -target — narrows the plan to one address, which makes the rest of the diff vanish from the output rather than from reality. HashiCorp documents it as a troubleshooting tool for exactly this reason.

Ban all three in the classifier, as a hard Class D on the diff of the configuration itself, not as a prompt instruction. This is the same lesson as "never let the agent delete an assertion to make the suite pass" from patch generation and test-driven loops: when the reward is a green signal, every path to green is in scope, and the cheapest paths are the ones that break the signal.

The decision the agent must not take is drift reconciliation direction. When reality and code disagree, there are two repairs — change the code to match reality, or apply the code and revert reality — and they have opposite consequences. One of them ratifies an emergency fix somebody made at 3am; the other undoes it, possibly during the incident it was fixing. No amount of context tells the model which. Have the agent produce the drift report, propose both diffs, and stop; a human picks the direction. This is the boundary that DevOps and SRE agents draws around the same class of judgement.

STEP 6

Ship it on an environment ladder, and measure the one number that is not vanity.

Roll out by consequence, not by capability. The order that works:

  • Plan-only, everywhere. The agent opens pull requests and the classifier labels them; nothing applies automatically. You are measuring the classifier and the agent's plan quality against real work, at zero risk, and this stage is where you find out that a third of your repository does not plan cleanly today.
  • Auto-apply Class A in ephemeral environments. Preview stacks, test accounts, anything you can delete and rebuild. A wrong apply costs a rebuild.
  • Auto-apply Class A in production; everything else queued. Only after the previous stage has produced weeks of Class A applies with no rollback, and only with the kill switch and the staged-rollout discipline from rollout and versioning already in place.

Instrument two numbers and ignore the rest. Class A rate — the share of agent runs whose plan lands in the auto-apply class with no human edit to the code — is the honest measure of whether this saves anyone time; a run that reaches a Class C plan has generated review work, not removed it. Applies requiring rollback is the safety number, and it should be countable on one hand, because the recovery story for infrastructure is worse than for application code: an apply that destroyed a resource is not revertible by reverting the commit, and you are into the manual, judgement-heavy repair described in repairing what the agent already did. Plans generated, pull requests opened and lines of HCL written are all vanity.

If you build one thing from this page, build the plan classifier. The agent is a swappable component that produces plans; the classifier is the thing that decides whether a plan can proceed without a human, and it is the only part of this system whose correctness you can actually establish. With it, an infrastructure agent is a well-bounded automation whose worst realistic outcome is a rejected pull request. Without it, you have moved the cloud's destroy button behind a language model and put a tired reviewer in front of it — and the reviewer's error rate, not the model's, is what will set your incident count.

Related: dependency upgrade agents for the same evidence-policy argument on a different diff, background coding agents for the review-queue arithmetic, and human in the loop for where an approval gate earns its latency.