Vulnerability Remediation Agents

The queue is not the work. Most of it should never reach the agent.

A remediation programme starts by conceding that the scanner backlog is mostly noise, and that an agent pointed at the whole backlog industrialises the noise. The base rate is the argument: analysis of exploitation data by Cyentia and FIRST puts the share of published CVEs ever observed exploited in the wild at around six per cent. Patch everything and roughly nineteen of every twenty patches you review were never going to matter — except that each one still costs review, still risks a regression, and still consumes the reviewer's willingness to look at the twentieth carefully.

Filter before you generate, in this order, because each stage is cheaper than the one after it:

• Reachability, not presence. A vulnerable dependency in your lockfile is not a vulnerability in your application unless the affected function is actually called. Call-graph reachability is the single largest reduction available — vendors selling it claim north of ninety per cent noise reduction, which you should read as a marketing figure and still take seriously, because the direction is not in dispute. This is the same discipline that makes dependency upgrade agents tolerable.
• Exploitation likelihood, not severity. CVSS scores how bad it would be; EPSS estimates whether anyone is doing it, and CISA's KEV catalogue records that someone demonstrably is. A KEV entry in reachable code is the top of your queue by a wide margin, and the "critical" with an EPSS in the low single digits behind an unreachable code path is not urgent no matter what colour the dashboard paints it.
• Blast radius, from your own architecture. The scanner does not know that one service holds customer data and the other renders a status page. This is the input no tool supplies and the one that most changes the ordering.

Decide what "fixed" means before you generate anything, because the obvious oracle is wrong.

The natural validation is: the functional test suite passes, and the proof-of-concept exploit no longer works. It is also demonstrably insufficient. The PVBench study of automated vulnerability repair evaluated patches that passed exactly that bar and found that over forty per cent of them failed under strengthened tests — variants of the same exploit rather than the specific input the agent was optimising against. It also found that for roughly twenty-eight per cent of the vulnerabilities studied, every patch produced was a false positive: nothing correct was generated at all, while the validation kept saying otherwise.

The failure is not mysterious. A model handed one exploit input will reliably produce a patch that rejects that input. It filters the string, adds the bounds check on the path the PoC took, special-cases the payload. The vulnerability is a class; the PoC is one member of it; and every part of the loop — the generation, the validation, the reward — points at the member.

So write the oracle to test the class:

• Mutate the exploit, then re-run. Change the offsets, the encoding, the field order, the path that reaches the sink. A patch that survives only the original input has told you it pattern-matched.
• Keep a functional-equivalence check alongside it. The most common way to defeat a security test is to break the feature.
• Refuse to grade on the model's own tests. An agent that writes both the fix and the test that certifies the fix is grading its own homework, which is the trap dissected at length in test-generation agents.

Build the reproduction first. It is the deliverable; the patch is a by-product.

Invert the intuitive order. The first job the agent is given is not "fix this" but "make this fail" — stand up an environment where the vulnerability demonstrably triggers, and commit that as a test. Everything good follows from having it, and the same inversion is what makes debugging and triage agents trustworthy rather than plausible.

• It makes the eval criterion objective. Fails before, passes after, in CI, on a machine that is not the agent's. No judge model, no "looks right to me."
• It converts an advisory into a regression test that outlives the patch. The lasting artefact of a remediation programme is a suite that would catch the bug coming back — and it does come back. Project Zero's analysis of in-the-wild zero-days found that a large minority were variants of previously patched issues, which is what an incomplete fix looks like from the outside, months later.
• It gives you an honest negative. "Could not reproduce" is a legitimate outcome that ends the run cheaply. Without it, an agent that cannot reproduce the bug will produce a patch anyway, because producing a patch is what it was asked to do.

Run the reproduction in a disposable, network-denied sandbox — you are executing exploit code, sometimes attacker-authored, in your own infrastructure, and the containment requirements are those in sandboxing and safe execution. Public infrastructure exists for the fuzzing-derived case: corpora that package real vulnerabilities as reproducible container images, and benchmarks built on them that ship both the pre-patch and post-patch trees, which is exactly the shape a proof-of-fix harness needs.

What the evidence actually says about the generation step.

The best-powered public study to date generated roughly six thousand patches for recently disclosed CVEs chosen to sit after the models' training cutoffs, using two frontier reasoning models at high effort. The distribution is the number to design around:

• ~26% resolved the vulnerability without materially changing behaviour — the outcome you actually want.
• ~20% fixed it but changed application behaviour — a patch that trades a security bug for an availability bug, and the one your security tests will happily wave through.
• ~54% failed to fix it, introduced a new vulnerability, or both, with about half of all patches leaving at least one existing exploit path open.

Two second-order findings matter more than the headline. First, guidance dominates: fix rates ran around 65% when the model was given correct guidance about the vulnerability, ~50% with none, and collapsed to ~15% with incorrect guidance. An agent that is confidently wrong about the root cause is materially worse than an agent told nothing — which means your triage step is not a nicety in front of the generator, it is the generator's largest single input, and a bad automated root-cause summary is actively harmful.

Second, the same gap shows up in competition conditions. In the DARPA AI Cyber Challenge finals, the cyber-reasoning systems discovered 86% of the synthetic vulnerabilities in scope and produced patches for 68% of that full set — four in five of the ones they found. Against real open-source code the ratio was worse: eighteen vulnerabilities discovered, eleven patched. Finding is ahead of fixing, consistently, across every setting anyone has measured — so a pipeline whose throughput assumption is "found equals fixed" is mis-sized from the start.

Ship the evidence bundle, not the diff.

What arrives at the reviewer determines whether this programme adds capacity or adds queue. A pull request containing a two-line change and the sentence "fixes CVE-2026-xxxxx" transfers the entire verification burden to a human who now has to reconstruct everything the agent already knew. Make the run emit a fixed bundle instead:

• The reachability evidence. The call path from your entry point to the vulnerable function, so the reviewer can see why this one was in the queue at all.
• The reproduction. The committed test, with CI showing it red on the parent commit and green on the head.
• The variant results. Which mutations of the exploit were tried and that they also fail. This is the section that distinguishes a fix from a filter, and it is the section a human would never have written by hand.
• The behaviour delta. What else changed — new error paths, rejected inputs that used to be accepted, performance on the hot path. The 20% "fixed but different" bucket lives here and nowhere else.
• The root-cause statement, and its confidence. Two sentences, plus an explicit "low confidence" when the agent could not tie the fix to a mechanism. Given the guidance sensitivity above, an agent that flags its own weak diagnosis is worth more than one that always sounds sure.

Grade the bundle mechanically before a human sees it, exactly as patch generation and test-driven loops prescribes: a missing reproduction or a failing variant test should bounce the run back to the agent, not into review. And do not let the agent choose the disclosure path — the systems furthest along here, including the ones operated by the labs that build the models, route every finding through a human before it is reported or upstreamed. That is a policy, not a capability gap, and it is the right one.

The merge queue is the throttle, and pointing this at other people's projects is a different programme entirely.

Everything above assumes the patches land in code you own. Aim the same machinery at your dependencies and you inherit a problem the open-source ecosystem spent 2026 documenting: curl closed its bug bounty in January after its confirmed-vulnerability rate fell from above fifteen per cent to under five, having received twenty submissions and zero real vulnerabilities in the first three weeks of the year; HackerOne paused the Internet Bug Bounty in March, stating plainly that discovery is no longer the bottleneck and that bounties do not fund remediation. The maintainers' complaint is not that machines find bugs. It is that machines send findings without the work that makes a finding actionable.

Which sets the rule for upstream contribution: send a patch and a reproduction, or send nothing. Disclose that the work was machine-generated, and hold the report until a human has verified the reproduction — the same gate you would want applied to reports arriving at you.

Instrument the programme on two numbers and ignore the rest. Merged-with-evidence rate — the share of opened pull requests that land carrying a passing reproduction — tells you whether the agent is producing verified fixes or plausible ones. Time from reachable-and-exploited finding to merged fix tells you whether any of it reached production. Patches generated, findings triaged and CVEs closed are all vanity: an agent opening twelve remediation PRs a day against a team that merges three has added a queue, per background coding agents.

Related: security operations agents for the detection-and-response half of the same team's workload; vulnerability management for agent platforms for patching the agent infrastructure itself; and code review agents for the precision-over-recall argument that governs every bot that files findings for humans.