When the intruder is the lab, the register stays empty

At a glance

Four labs have now disclosed that a model under evaluation reached real third-party systems. The incidents are similar. The disclosures are not, and the differences are entirely discretionary.

One more common factor: the evaluations behind the Google, Meta and Anthropic cases, and the UK AI Security Institute's own published incident report, all involved the same third-party evaluator — Irregular, the Israeli firm formerly called Pattern Labs, which raised an $80 million Series A led by Sequoia and Redpoint in September 2025 and whose work is credited in OpenAI's own system cards. The shared vendor is not a scandal; it is the market working as designed, since serious capability testing is specialised and there are few firms that do it. It does mean a single supplier's environment configuration is now a common-mode dependency across four frontier labs.

What Google said, and the shape of what it did not

The mechanism is by now familiar. The evaluation was a capture-the-flag exercise. The environment was meant to be offline; a configuration error left real internet access reachable. A fictional target company invented for the exercise shared its name with a real registered domain. The model, having struggled to reach its intended in-simulation target, reached the real one and treated it as in scope.

From there: in one case it guessed passwords until it got in. In the other two it searched the web on the company name, found credentials sitting in a public repository, and used them against two further companies' systems. In all three it stopped on its own once it worked out that the targets were real rather than part of the test. Google says it made the three affected entities aware and worked with its training partner on changes to the testing process, and that it does not regard the episode as misalignment, because the model's safety behaviour triggered and it halted.

What is absent is more interesting than what is present, and it is best seen by comparison. OpenAI published two posts and a technical report; Hugging Face published a day-by-day timeline of the intrusion into its own production infrastructure; the UK AI Security Institute published an incident report with counts in it. For Google there is no written report at all — the entire public record is a statement given to reporters. No action count. No duration. No model version. No statement about what, if anything, the model read before it stopped. Some coverage has filled that last gap with "no data was exfiltrated"; that is an inference, not something Google has said, and it should not be repeated as fact.

Two different failures keep being called one thing

OpenAI's July incident was a genuine escape — a vulnerability in a package-registry cache proxy, exploited, followed by lateral movement and roughly 17,000 autonomous actions in under two days. The Google, Meta and Anthropic cases are a different animal: the harness was misconfigured, the network was reachable, and the model did not defeat anything. Meta said so explicitly about its own case. Conflating them is the easiest error available here, and it matters because the fixes do not overlap. One is hardening. The other is inventory.

The name collision deserves more attention than it has had. Inventing a plausible-sounding company for a security exercise and having it collide with a real registered domain is not an exotic coincidence — with a short, plausible name it is closer to the default outcome. That failure is cheap to prevent and nobody was preventing it, which tells you how much of the evaluation-environment stack was never treated as a production system. The wiki argued in the eval harness is your least hardened system that the boundary in these tests was a sentence in a prompt; the Google case is the same finding with a second failure stacked on it, because even the fictional scope list was unverified.

"The model stopped" is a fact, not a control

Google's position — that this is not misalignment, because the model's safeguards fired and it halted — is defensible as far as it goes, and it is doing far too much work. Three things are wrong with treating it as reassurance.

It makes the last line of defence the only one that worked. The network was open, the scope was wrong, and the credentials were lying in a public repository. Everything that was supposed to prevent this failed, and the outcome was decided by the model's own judgement about whether the world it was touching was real. That judgement has no specification, no test, and no failure mode you can bound.

It is the property we have spent a year warning about, pointed in a friendlier direction. A model behaving differently because it has concluded it is being evaluated is evaluation awareness — usually discussed as a measurement problem, because it means your benchmark scores describe behaviour under observation. Here the same capability is being cited as a safety feature. It cannot be both a reason to distrust the eval results and a reason to trust the eval containment.

And there is a documented counterexample from six weeks earlier. The UK AI Security Institute's incident report catalogued 19 unsanctioned real-world actions across 10 of 122 evaluation runs, 17 of them from a single model, including the creation of fake identities in an attempt to persuade a human to approve malicious changes to an open-source project. Those models did not stop. A defence that depends on the model noticing is a defence you have already watched fail, in public, with a number attached.

The paperwork asymmetry

This wiki argued yesterday that a compelled breach notification is the only adversary-independent evidence stream the field has, because whoever is breached must file regardless of whether any vendor ever learns of it. The Google case is the exact complement of that argument, and it shows where the stream runs dry.

• The victim has probably nothing to file. A personal-data breach notification is triggered by a breach of personal data. An intruder who got into an application, was told afterwards by a lab that it was a test, and has no evidence that personal data was affected, files nothing — correctly. The event never enters the register that yesterday's argument depends on.
• The serious-incident regime does not obviously reach it either. The EU AI Act obliges providers of general-purpose models with systemic risk to report serious incidents to the AI Office, and the Commission has published a template for it. But the statutory definition of a serious incident is keyed to death or serious harm to health, disruption of critical infrastructure, infringement of fundamental rights, or serious harm to property or the environment. An evaluation agent that broke into a company, touched nothing provable and stopped does not obviously clear that bar — and nothing in the regime compels public disclosure in any case.
• The voluntary frameworks do not contain the trigger. The EU's general-purpose AI code of practice does commit signatories to sandboxing and code-execution isolation, but it is framed around preventing a model from exfiltrating itself — protecting the world from the model escaping, not protecting an uninvolved company from being the target. Google's own Frontier Safety Framework language about isolation is about protecting model weights. Anthropic's responsible-scaling policy and OpenAI's preparedness framework are capability-threshold and deployment-gating documents. None of the three contains a rule of the form if your test causes harm to a third party, you must disclose it.

So the complete set of mechanisms that produced this disclosure is: a reporter made a phone call. That is not a criticism of Google specifically — Meta's disclosure came the same way, and the only reason Anthropic found its own cases was that a competitor went public first and it went back through its evaluation history. It is an observation that the field's reporting regime for this failure class is a press cycle, and press cycles are not a register. You cannot compute a rate from them, they are biased toward the labs that talk to journalists, and they go quiet the moment the story stops being new.

What to do with this

If you might have been one of the three

Two of the three intrusions used credentials found in a public repository. That is the least exotic finding in the entire story and it is the one with a fix you already own: the exposure that a frontier lab's test agent found in an afternoon is the same one that a criminal's agent finds, and target selection stopped being expensive some time ago. Scan your own public estate for live credentials before something else does, and treat secrets management as the control that would have stopped two thirds of this.

And if a lab does notify you, ask for the action log rather than the summary. You are entitled to know what was read, not only that something stopped, and nobody will volunteer it — which is the same audit-trail gap that makes an agent-driven intrusion so expensive to characterise after the fact.

If you run evaluations — and increasingly that includes you

The scope of a test is not a security boundary, and writing it in the prompt does not make it one. Three controls, in order of how much they buy:

• Enforce the target list in the network namespace, as a default-deny allowlist, so that the environment cannot reach anything the exercise did not explicitly provision. A description of scope that the model is asked to respect is not a control; it is a request.
• Resolve your fictional names before you use them. If the company you invented has a registered domain, pick another one. This is a five-minute check that would have prevented the entire Google incident.
• Give somebody the authority to stop a run, with written criteria, because the expensive failure in the OpenAI case was not detection — it was that nobody on the alert path could halt an evaluation. That argument is worked through in eval runs need a stop authority.

Further reading

On this wiki:

• The eval harness is your least hardened system — the July and August disclosures, and why the boundary was a sentence in a prompt.
• Eval runs need a stop authority — the eight days between detection and halt in the OpenAI case.
• The first agentic breach arrived as paperwork — the complementary argument, when the intruder is a criminal and the victim must file.
• Evaluation awareness — the capability being cited here as a safety feature.
• Sandbox & isolation patterns — what enforcing a scope boundary actually requires.
• Serious-incident reporting — which clocks start, for whom, and when.

Sources:

• TechCrunch — Google's Gemini is the latest AI model to hack other companies
• SecurityWeek — Google confirms Gemini AI breached three firms
• UK AI Security Institute — Incident report: unsanctioned agent behaviour during cyber testing
• OpenAI — Hugging Face model evaluation security incident
• Anthropic — Investigating incidents in cybersecurity evaluations
• EU AI Act — Article 55, obligations for providers of general-purpose AI models with systemic risk