Debugging & Triage Agents

Split the job at the reproduction line.

A bug report arrives as a description of a symptom. A patch is a change to a cause. The step between them is the one that carries all the risk, and it is the step that a fix-first agent skips entirely.

• Phase one produces a repro; phase two produces a fix. Run them as separate stages with separate exit conditions. The exit condition for phase one is machine-checkable in a way almost nothing else in agent work is: a test that fails on the current commit and describes the reported symptom.
• A phase-one failure is a good outcome, not a broken run. "I could not reproduce this, here is what I ruled out and what I would need" is genuinely valuable and is what a competent engineer reports at hour two. An agent that cannot return this will fabricate a cause instead, because you gave it no acceptable way to fail.
• The repro is the artifact that survives. The patch gets rewritten in review; the failing test goes into the suite and prevents the regression forever. Design your output schema around the test, with the diff as an attachment rather than the headline.
• This is why patch generation and tests is a different playbook. That one assumes you know what to change. This one is about the phase where you do not, and the two have almost opposite cost profiles.

Give it the three inputs a coding agent never needed.

Code-writing agents work from source. Debugging works from evidence about a specific execution, and source is only one of its three inputs. Missing either of the other two is what forces the model to guess.

• Runtime state. The values, not the variable names. A stack trace is a skeleton; what the agent needs is the request that produced it, the arguments at the failing frame, the relevant rows, the feature flags in effect, the config that was actually loaded. Most teams have this in their tracing already and never connect it to the agent.
• History. When did this start failing, and what changed around then. Deploys, migrations, dependency bumps, config edits, traffic shifts. A bug that started at 14:03 on Tuesday has a much smaller suspect set than a bug with no time signature, and this is the single input that most reliably collapses the search.
• The ability to re-run. Non-negotiable, and the reason a debugging agent needs a genuine execution environment rather than a code index. It must be able to change one thing and observe the result — which means a checked-out repo, installed dependencies, and a way to reach representative data. Reuse the isolation model in sandboxing and execution; the requirements are the same, the session is just longer.

Make the loop narrow the space, not propose fixes.

Left unstructured, a model debugs by generating the most plausible cause and checking it — a strategy that works on common bugs and degrades to random search on the ones you needed help with. Bisection is the algorithm to impose.

• Every step must eliminate something. Require the agent to state, before each action, which hypothesis the result will kill. An action whose outcome cannot rule anything out is a wasted step, and a run made of those is the "flailing" everyone recognises from tool error recovery.
• Keep an explicit, written suspect list. Hypotheses with status: open, eliminated, confirmed — carried in the working state, not left implicit in the transcript. Without it, the agent re-tests things it already ruled out fifteen turns ago, which is the dominant failure mode of long debugging runs.
• Bisect over commits, and also over inputs and config. git bisect is the famous one and often the fastest path when you have a version where it worked. The same discipline applies to shrinking an input until the failure disappears, and to flipping config one flag at a time.
• Instrument rather than infer. Adding a log line and re-running beats another paragraph of reasoning about what the value probably is. Agents under-use this because printing feels like a non-answer; make it a first-class tool and reward its use.
• Cap the loop on eliminations, not on turns. Stop when the suspect list stops shrinking. Turn limits cut off productive runs and let unproductive ones spend the full budget; elimination rate distinguishes the two directly.

Triage is a different agent, and it runs first.

Most of what arrives in a bug queue does not need debugging. It needs routing — and mixing the two jobs into one agent produces something that debugs duplicates at full price.

• Triage answers four questions and stops. Is this a duplicate? How bad is it? Which component and which owner? Is there enough information to act? It is cheap, high-volume, and should be a small fast model with retrieval over the issue history — a good fit for the pattern in small and local models.
• Deduplication is the highest-value output by a distance. A queue where the same crash arrives forty times is the normal case, and collapsing it is both the biggest cost saving and the biggest signal gain — forty reports is severity information the first report did not carry.
• Severity is a policy, not a judgement. Give the agent the rubric your team actually uses, with examples. Left to its own priors a model will rate everything medium, which is the same as rating nothing.
• "Not enough information" is a real, common, useful verdict. An agent that asks the reporter one precise question — the version, the exact input, whether it reproduces in a fresh session — recovers more bugs than one that speculates, and it does it for a hundredth of the cost.
• Only promote to the debugging agent on an explicit gate. Reproducible-in-principle, owned, not a duplicate, above a severity floor. Everything else waits for a human or for more reports. Without this gate your debugging spend is set by your queue volume rather than by your bug volume.

Production access: read everything, run nothing.

Debugging pulls agents toward production, because that is where the evidence is. This is the sharpest security decision in the playbook and it deserves an explicit line rather than a gradual drift.

• Reads from production, writes only in the sandbox. Traces, logs, metrics, schema, query plans, a redacted sample of the failing payload — all fine, all read-only. Attaching a debugger, running a query that takes a lock, restarting a service, editing a flag: not from this agent. Enforce it with credentials, not with instructions; see scoped credentials for agents.
• The failing payload is the most sensitive object in the system. It is real customer data by definition, and the agent wants to put it in a prompt. Redact at the boundary, before it reaches the context, and log which fields were redacted so the agent knows something was there.
• Bug reports are untrusted input. A stack trace, a pasted log, an attached file and a reporter's description are all attacker-controllable on any product with external users. This is the same exposure as security-operations agents, and the same rule applies: content from the report is evidence about the world, never an instruction.
• Separate the incident path from the bug path. During an outage, people want the agent to act. That is a different risk posture with different approvals and belongs in incident response for agents — not something you reach by loosening the permissions on your bug-queue agent.

Evaluate on reproduction, not on merge rate.

Merge rate is the metric everyone reaches for and it measures the reviewer as much as the agent. The reproduction split gives you something better.

• Reproduction rate. Share of accepted bugs where the agent produced a test that fails on the buggy commit. Objective, cheap to check, and it moves for real reasons.
• False-repro rate. The one that matters most and the one nobody tracks: tests that fail for a reason unrelated to the reported symptom. This is the failure that costs you the most trust, because it looks exactly like success. Sample and check by hand.
• Correct-abstention rate. How often the agent said "cannot reproduce" on bugs a human also could not reproduce. If this is near zero, your agent has no way to fail honestly and every other number is inflated.
• Eliminations per run and cost per reproduction. The efficiency pair. Cost per reproduction is the number to compare against an engineer's hour; cost per merged patch flatters you by hiding the runs that produced nothing.
• Build the eval set from your own closed bugs. You have the ideal corpus already: the bug, the commit that fixed it, and the test that came with it. Check out the parent commit and you have a graded exercise with a known answer — far more predictive for your codebase than a public benchmark, and the construction is described in evaluating coding agents.

Related: coding agent architecture for the harness this sits in, repo navigation and context for finding the code once you know where to look, code review agents for the reviewer on the other side of the patch, and trajectory evaluation for grading the search rather than the answer.