Documentation Agents

Every other coding agent has an oracle. This one doesn't.

A patch agent gets a test suite. A migration agent gets a compiler. A review agent gets a human who either merges or doesn't. Each of those is a signal that says wrong before the change reaches anyone. Documentation has no such signal, and the gap is not a tooling deficiency you can close — it is what documentation is for. If the correctness of a sentence could be derived from the code, the sentence would not need writing.

• The failure is silent and durable. A broken build is discovered in ninety seconds. A page that describes a flag removed last quarter is discovered by the fourteenth person who tries it, months later, and most of them never tell you.
• Fluency is the specific hazard here. Models are better at sounding like documentation than at being right, and readers use register as a proxy for authority. A confident wrong paragraph outcompetes a hedged correct one — the grounding problem in hallucination & grounding, aimed at the artefact readers trust most.
• So "review it before merge" is not a plan. It is the plan everyone writes down and nobody executes, because reviewing prose for factual accuracy is slower than writing it, and a docs PR arriving weekly gets skimmed by the third week.
• Which fixes the design question. Not "how do we get the agent to write good docs" but "which claims can be mechanically falsified, and how do we forbid the agent from making any others unsupervised."

Split the corpus by verifiability, and let that split decide authority.

Your docs are not one thing. They are four things with wildly different truth conditions, and the useful move is to sort every page into one of them before you write a line of agent code.

• Derived reference — machine-checkable, agent owns it. Parameter tables, return types, error codes, CLI flags, environment variables, endpoint schemas. Truth is a function of the source, so drift is detectable and a regeneration is verifiable. This should be generated, not written, and the agent's job is to keep the generator honest when the source shape changes.
• Executable prose — machine-checkable, agent owns it. Code samples that compile and run, quickstarts that a container can execute end to end, links that resolve, version numbers that match the lockfile. Anything here is a claim with a test attached; wire that test into CI and the agent inherits an oracle for this slice.
• Explanatory prose — not checkable, agent proposes only. Why this component exists, what it replaced, which of three approaches you should pick, the failure mode that motivated the design. The value of this text is that a human knew something the source does not record.
• Obligations — not checkable, never automate. Security guidance, migration steps, data-handling instructions, anything a reader will follow without thinking. A wrong sentence here is an incident, not a typo.

Docs written from source inherit the source's assumptions, bug included.

The obvious build is to point the agent at the code and ask for documentation. It produces something that reads well and is worth very little, for a reason worth being precise about: it is a lossy restatement of the input, and every interesting sentence in good documentation is information the input does not contain.

• It cannot tell you intent. The code says the retry count is 3. Documentation's job is to say why 3, what happens at 10, and that the number was chosen for a downstream rate limit that no longer exists.
• It launders bugs into specification. When an agent describes behaviour by reading the implementation, a defect becomes documented behaviour — and once documented, it becomes a compatibility obligation someone will refuse to break.
• It cannot say "don't use this". The most valuable sentence in a mature doc set points away from itself: deprecated, superseded, only for the legacy path. Nothing in the source distinguishes the recommended API from the one that still works.
• Give it the other inputs instead. Commit messages, pull-request discussion, issue threads, design docs, and support tickets are where intent lives, and retrieving over them is the actually valuable capability — the retrieval problem framed in repo navigation & code context.
• Make it cite. Every non-derived claim carries a link to the commit, issue, or thread it came from. An uncited assertion from a docs agent costs a reviewer more to verify than to write from scratch, which is the same economics as in dependency upgrade agents.

Make deletion a first-class output, because nothing else will.

Left alone, a docs agent monotonically grows the corpus. Every run adds; no run removes. Two quarters in you have a doc set 40% larger, with the same true content diluted across more pages, more internal contradictions, and worse search results — and the agent's dashboard says it has been productive throughout.

• Contradiction is worse than absence. Two pages describing the same endpoint differently is a strictly worse state than one page, because now the reader must adjudicate and has no basis to. Detecting these pairs is a genuinely good use of a model and something no linter does.
• Give the agent a delete lane with its own evidence bar: the documented symbol no longer exists, the page has had no inbound traffic in two quarters, its content is a strict subset of another page, or it references a version below your support floor. Each of those is checkable.
• Redirect, don't orphan. A deletion PR that leaves dangling links is a net loss; the agent's deletion is only complete when every inbound reference is repointed, which is exactly the mechanical work it is good at.
• Budget the corpus. If page count is allowed to rise every quarter, nothing forces the trade-off. A ceiling turns "should we add this page" into a real question, and it is the single cheapest control on a docs agent.

Point it at where readers actually failed.

Absent a signal, the agent will improve whatever is easiest to improve, which is the well-maintained page a human already cared about. The pages that need work are the ones nobody has opened in a year and are wrong. You have to supply the direction; it is not derivable from the repository.

• Support tickets are the highest-grade input you own. Every ticket answered with "that's in the docs, here" is a discovery failure; every ticket answered with a fact that is not in the docs is a coverage hole with the correct wording already written in the reply.
• Search queries with no result, and searches followed by no click. These name the vocabulary gap between what readers call the thing and what you call it — usually the cheapest documentation fix in existence.
• Diffs that touch behaviour but no docs. A pull request that changes a default, an error message, or a public signature and updates nothing under docs/ is a staleness event you can catch at the moment it happens rather than a year later.
• What your own coding agents got wrong. When an internal agent misuses your API, the transcript is a documentation bug report from an unusually literal reader — the same feedback loop that tool discovery & docs describes for tool descriptions, applied to human-facing pages.

Operate it: small PRs, scoped credentials, and a metric that can go down.

The operational failure is not a bad page, it is volume nobody reviews. Everything here is aimed at keeping the human cost per change low enough that review actually happens.

• One concern per pull request. A regenerated reference table is auto-mergeable; a rewritten conceptual page is not; a deletion is not. Mixing them forces the strictest review bar onto the whole batch and the batch stops moving.
• Run derived reference on every merge, prose work on a weekly cadence. Reference drift should never survive a commit; explanatory work should arrive at a rate your reviewers can absorb, per scheduled & triggered agents.
• Scope the credentials. Read the repo, read the ticket system, push a branch, open a PR. Not merge, not publish to the docs site, not touch the redirect map outside the branch. Docs sites are public surfaces and a compromised docs agent is a content-injection channel into your own users' browsers.
• Treat retrieved text as untrusted. Issue threads and support tickets are attacker-writable in most products, and this agent reads them with repo write access nearby — the containment argument from sandboxing & safe execution applies without modification.
• Report freshness and reader-failure rate, never pages written. Pages written only goes up, and it goes up fastest when the agent is doing the least valuable work. Median age of derived reference relative to its source, and tickets-caused-by-docs per month, both go down when the thing is working and up when it isn't.

Related: code review agents for the reviewer on the other side of these PRs, evaluating coding agents for measuring any of it, and translation & localization agents once the corpus needs to exist in more than one language.