Spec-Driven Development with Coding Agents

The loop cannot check itself, and that is the whole argument.

A coding agent's inner loop is genuinely good at self-correction: write a patch, run the tests, read the failure, try again. What it cannot do is notice that it solved a different problem than the one you had. Its tests encode its interpretation. Its plan restates its interpretation. Its summary describes what it built, accurately, against an interpretation nobody checked. The loop converges — on the wrong target, with every internal signal green.

This is why "the agent wrote tests and they pass" is a weaker claim than it sounds, and it is a different weakness from the flaky-and-overfit failures catalogued in patch generation and tests. Those are failures of the oracle's rigour. This is a failure of the oracle's independence: same source, same blind spot.

A spec supplies an oracle from outside. It is written before the agent starts, by a human who owns the outcome, in terms of what the system must do rather than how. Then the agent's interpretation has something to be wrong against. That is the only job the document has, and every rule below follows from it.

Specify observable behaviour; everything else belongs somewhere other than the spec.

The most common way a spec fails is by filling up with implementation. It reads well, the agent follows it obediently, and it has quietly become a plan — an artefact that cannot contradict the code because it is the code, in prose.

What belongs in it:

• Observable behaviour. Inputs, outputs, and the mapping between them, stated so that someone who cannot see the source could tell whether it holds.
• Contracts at the edges. Endpoint shapes, schemas, error codes, wire formats, event payloads — the things other people's code depends on and that a refactor must not silently change.
• Invariants. What must be true after every operation. These are the highest-value lines in the document because they constrain implementations the author never imagined.
• Error and edge behaviour. What happens on a duplicate submit, an empty list, a timeout mid-transaction, a permission the caller lacks. Agents are systematically weak here, because ordinary training data over-represents the happy path.
• Explicit non-goals. The single highest-yield section and the one everyone omits. An agent with a plausible adjacent improvement in reach will take it, and "do not touch the auth middleware" is a line that pays for the whole document the first time it holds.
• Acceptance criteria. The list something can be checked against — see the next step.

What does not belong: file names, class hierarchies, algorithm choices and library picks. Those are the agent's job and, more importantly, they are the part you want it free to revise when it discovers the codebase disagrees with your mental model — which is the strength described in coding agent architecture. Also excluded: anything you cannot check. A criterion like "the code should be maintainable" is not a weak criterion, it is not a criterion; it consumes review attention and constrains nothing.

Every acceptance criterion gets a check id or a named human.

This is the step that separates spec-driven development from spec-shaped ceremony. Go through the acceptance criteria one at a time and assign each one an executable check — a test name, a script, a lint rule, a migration verification query. Criteria that cannot be made executable get the name of the person who will verify them by hand, in the document.

AC-1  Duplicate submit within 60s returns the original receipt
      -> test_idempotent_submit_returns_original
AC-2  Expired token yields 401 with code=token_expired, never 500
      -> test_expired_token_401
AC-3  No endpoint returns a raw provider error body
      -> lint: forbid-provider-passthrough
AC-4  Migration is reversible on a populated table
      -> scripts/verify_down_migration.sh
AC-5  Onboarding copy reads clearly to a first-time user
      -> MANUAL: review by product owner

Two numbers fall out of that table and both are worth watching: how many criteria have a check, and how many manual items are outstanding. A spec where nine of twelve criteria map to nothing is not driving development, and now you can see that rather than sensing it.

The ordering matters as much as the mapping. Write the criteria before the agent starts, and let it write the tests that satisfy them — that sequence keeps the intent human-authored while the labour stays automated, which is the split that makes test-generation agents safe to use. Reverse it and you get tests that assert whatever the implementation happens to do.

The plan is a separate artefact with a different lifetime — review it, then throw it away.

Agents produce plans, and plans are useful. They are just not specs, and conflating the two loses both.

The spec is authored by a human, states intent, is versioned with the code, and survives the change. The plan is authored by the agent, states an approach, and is disposable the moment the work merges. They also earn their keep at different moments: the spec catches "you built the wrong thing" and the plan catches "you are about to build it the wrong way".

Reviewing the plan is the highest-leverage five minutes in the whole workflow, because it is the last cheap moment. A plan that says it will refactor the session store before adding the field is telling you, for free, about a two-day review you did not want — and the fix is one sentence, versus a rejected diff after the work is done. This is the same economics that makes plan-and-execute worth the extra round trip, applied to the human in the loop rather than the model.

Two practical rules. Keep the plan out of the repository — a merged plan becomes a stale document that the next agent reads as current design. And when the plan and the spec disagree, that is a signal to stop and reconcile, not an inconsistency for the agent to resolve on its own; it usually means the spec was ambiguous, and the ambiguity is now cheap to fix.

Spec drift is the failure mode, and a stale spec is worse than none.

Documents rot. That is survivable when the reader is a person, who applies judgement and notices the file was last touched two years ago. It is not survivable when the reader is an agent, because an agent reads a stale spec with exactly the confidence it reads a fresh one, and it will regenerate the behaviour the document describes — undoing a deliberate change nobody wrote down. The stale spec is not neutral. It is an active instruction to revert.

Four rules keep it honest:

• The spec lives in the repository, next to the code it governs. Not in a wiki, not in a ticket, not in a doc tool the agent cannot read. If it is not on disk it is not in context, and if it is not in context it changes nothing.
• Behaviour changes and spec changes ship in the same pull request. Enforce it the way you enforce any other co-change rule: a CI check that flags a diff touching the module without touching its spec, requiring an explicit override. The override is the point — it makes the exception visible instead of silent.
• Date and scope every spec, and delete the ones you will not maintain. A spec covering a module nobody has touched in a year is a liability with no offsetting benefit. Deleting it is a real improvement to the repository's signal, and it takes a minute.
• Let the agent flag contradictions rather than resolve them. "The spec says X, the code does Y" is the most valuable output an agent can produce here, and it costs you one line of instruction. Agents will silently pick one otherwise, and it will be the one that makes the tests pass.

For work that spans sessions, agents or weeks, the spec is also the handoff. A background agent picking up the task on Thursday has no memory of Tuesday's conversation; the spec is the only context that persists, which is why this discipline pays for itself fastest in background coding agents and large-scale migrations.

Know where this pays, and stop doing it where it does not.

Spec-driven development is overhead with a return, and the return is not uniform. Applied everywhere it becomes ceremony, teams resent it, and it gets abandoned along with the cases where it was working.

It pays when:

• The work is greenfield. No existing behaviour to infer from, so the spec is the only oracle available. This is where the technique is close to mandatory.
• The contract outlives the implementation. Public APIs, event schemas, file formats, anything another team consumes.
• The change is large or long-running. Migrations, rewrites, anything crossing sessions or agents.
• Being wrong is expensive. Money movement, permissions, data deletion, regulated behaviour — where a plausible-but-wrong implementation is not caught by a user complaining.

It does not pay when:

• A failing test already is the spec. For a bug fix, the reproduction is a better, cheaper, executable statement of intent. Writing a document around it adds nothing.
• The work is exploratory. If you do not yet know what correct looks like, a spec written in advance is a guess that will constrain the agent toward your wrong guess. Prototype first, then specify what you decided to keep.
• The change is smaller than the document. Renames, dependency bumps, copy tweaks.

The anti-pattern to name plainly: spec ceremony as a substitute for a test suite. A repository with beautiful specs and thin tests is in a worse position than one with no specs and good tests, because the specs create a felt sense of coverage that nothing enforces. The spec exists to say what to check. The suite is what checks it — and the discipline of deriving one from the other is the same one described in eval-driven agent development, one level up.