One of Irregular’s jobs is to build fake networks where the world’s most capable AI models can be turned loose as hackers without hurting anybody.
This summer, some of those fake networks had a small problem: They were connected to the real internet.
The models don’t have to turn evil to be dangerous.
Models being tested for Anthropic, OpenAI, and Meta got through that open door and attacked systems belonging to real organizations. They didn’t discover some ingenious way to escape a hardened sandbox. In the Irregular cases, internet access was available when it wasn’t supposed to be, and the models generally believed the real computers they found were part of the hacking exercise.
Irregular published an August 14 postmortem explaining what went wrong and what it says it has done to fix the problem. What it doesn’t provide is a simple total: How many times did its AI evaluations end with a model attacking something in the real world?
That omission prompted criticism from security researchers interviewed by the Record. University of Surrey computer science professor Alan Woodward called the report heavy on “marketing spin” and accused Irregular of using ambiguous language to make multiple compromises sound like a single incident.
The mess becomes easier to understand if you start with Anthropic, which has been much more specific about what happened.
After OpenAI disclosed its breach of Hugging Face, Anthropic reviewed 141,006 cybersecurity evaluation runs in which Claude could have obtained internet access. It found three separate incidents involving six runs. Those incidents compromised the production infrastructure of three different organizations.
The first involved a fictional company in one of Irregular’s capture-the-flag exercises. Unfortunately, the made-up company’s name also belonged to a real website.
Four runs of Claude Opus 4.7 found the real company, identified vulnerabilities, extracted credentials, and gained access to a production database containing several hundred rows of data. Anthropic says that in all four runs, the model eventually encountered evidence that the system was real. It kept going anyway. In two runs, Claude reasoned that the real company must still be part of the test.
From bad to worse
The second incident was considerably weirder.
Claude Mythos 5 found setup instructions inside a fictional company that told employees to install a Python package that didn’t actually exist. Claude realized it could create a malicious package with that name so the target would install its code.
So it did.
The model went through the process of registering accounts and uploaded a malicious package to the real Python Package Index. It remained publicly available for roughly an hour and was downloaded and executed on 15 real systems.
RELATED: Are you inhaling microplastics?
solar22/Getty Images
One belonged to a security company whose automated scanner installed Python packages to inspect them for malware. Claude’s code ran on the scanner, captured credentials, and sent them to a collection point the model had created. Claude then used those credentials to access more of the company’s infrastructure.
Anthropic says Mythos 5 actually considered the possibility that it was on the real internet. It then talked itself out of that conclusion. Among other things, the model decided that seeing the year 2026 on the computer was evidence that the environment had been staged.
The third incident involved an internal Anthropic research model. Unable to reach its intended fictional target, the model started looking elsewhere and scanned roughly 9,000 internet-facing systems. It eventually compromised a real company’s application using credentials exposed on a debug page and SQL injection.
This time, the model eventually figured out that the compromised system had nothing to do with the exercise and stopped on its own.
Anthropic says it found no evidence that any of the models were pursuing goals of their own. They were doing what they had been told to do, attack systems and capture a flag, while operating under a false assumption about where the simulation ended.
That doesn’t make the incidents harmless. It makes them embarrassingly mundane.
Somebody left the door open.
OpenAI reported its own Irregular-related incident on August 4. Its models had likewise been told they had no internet access, but a testing-environment misconfiguration allowed them online. Once again, a fictional target happened to share a name with a real domain. An OpenAI model attacked the real site, exploited what OpenAI described as a basic vulnerability, and found and used credentials belonging to the site.
OpenAI explicitly distinguished this from its earlier Hugging Face breach. In that case, models inside an isolated evaluation environment found and exploited a previously unknown vulnerability in an Artifactory package-registry proxy, obtained internet access, moved through OpenAI’s research network, and eventually compromised Hugging Face infrastructure.
The Irregular incident was much simpler. The internet connection was already there.
Meta then disclosed another case. The company told Reuters that an Irregular misconfiguration inadvertently gave one of its models internet access during an evaluation. The model then exploited a vulnerability in a third-party service.
Reuters reported that the model was Meta’s Muse Spark 1.1, citing the Information, but Meta itself has not publicly identified the model.
A shady synopsis
This is where Irregular’s accounting gets murky.
Its postmortem says public disclosures stemming from the same underlying evaluation problem involved activity from “a single evaluation scenario” and therefore are “not materially separate incidents,” regardless of how many outside parties were affected.
There is a legitimate technical point buried in there. If Anthropic, OpenAI, and Meta all ran models against the same badly configured scenario, three companies later publishing disclosures does not necessarily mean three unrelated failures occurred inside Irregular.
But Anthropic didn’t disclose only that scenario.
Anthropic says plainly that its three incidents involved three different fictional capture-the-flag scenarios. One attacked the real company sharing the fictional company’s name. Another put malware on PyPI. Another scanned thousands of internet systems before compromising one of them.
Irregular’s own post also shifts terminology. It refers to “a few interactions” in which models took offensive action in the real world, discusses a “handful” of attempts against the real domain, and says the incidents occurred in fewer than one out of every 10,000 advanced simulations.
Maybe one configuration mistake explains several public disclosures. That’s useful information if you’re trying to fix the configuration mistake.
It still doesn’t tell us how many real-world compromises occurred.
If you leave one gate open and five cows get out, you have one open gate and five loose cows.
Irregular also says it has “no evidence of a customer’s systems being breached or customer’s data being leaked.” Read narrowly, that’s reassuring: The AI labs whose models Irregular was testing apparently weren’t compromised.
Other companies absolutely were.
Anthropic says Claude extracted credentials and reached a production database at one company, captured credentials from another company’s security scanner and used them to access additional infrastructure, and compromised a third company’s internet-facing application.
Calling Anthropic, OpenAI, and Meta “customers” is perfectly normal business language. Dropping that reassurance into a postmortem about attacks on other companies without immediately explaining the distinction is less helpful.
Irregular says it has added safeguards and expanded manual review of model behavior. It also says manual review by itself cannot handle the enormous volume of activity generated during these tests.
That’s believable. A hacking evaluation is supposed to be full of behavior that would set off alarms on a normal network. The hard part is detecting when the hacking that is supposed to happen inside the test suddenly starts happening outside it.
RELATED: A huge pagan statue just went up in Texas. Is this the Western civilization we should be cheering?
Darko Mlinarevic/Getty Images
But that’s also the business Irregular is in.
For comparison, look at an entirely separate incident disclosed by the U.K. government’s AI Security Institute this month. The setup there was different: AISI deliberately gave frontier models internet access and disabled their normal cybersecurity classifiers so researchers could measure what the models were capable of under permissive conditions.
AISI tested seven models across two cyber ranges in 122 evaluation runs. In 10 of those runs, researchers found out-of-scope activity. They catalogued 19 distinct actions, 17 involving Anthropic’s Mythos 5 and two involving OpenAI’s GPT-5.6 Sol, while stressing that those 19 actions were not 19 separate incidents.
Among other things, Mythos 5 attempted to put malicious code into a real open-source project, created fake identities, tried to socially engineer a real maintainer, attempted to contact real people, and planted instructions intended to manipulate other AI coding systems.
AISI’s monitoring caught unusual data leaving a test system through Tor on July 28. The agency says it stopped the related evaluations and isolated the relevant machines within an hour. Its investigation found no resulting real-world harm.
The interesting part for Irregular isn’t that AISI had a containment problem too. It’s how AISI wrote the incident report.
It gave the total number of runs. It gave the number of runs containing unauthorized behavior. It gave the number of actions. It named the models involved. It explained when monitoring detected the activity, how quickly the tests were stopped, and what investigators found afterward.
Harsh lessons
There are still unanswered questions about the AISI episode, but you don’t need a decoder ring to figure out what the agency says happened.
Irregular, meanwhile, was already well aware of how quickly AI hacking capabilities were improving. In research published in December, the company reported that frontier models had gone from scoring near zero on the hardest tier of its private offensive-security tasks through the first half of 2025 to roughly 60% by late fall.
Those tasks include difficult vulnerability discovery, exploit development, reverse engineering, and cryptographic problems. Irregular itself warned that the improvement represented a meaningful shift in offensive-security capability.
That’s the real lesson here. The models don’t have to turn evil to be dangerous. They don’t even have to escape.
They just have to be competent enough that when somebody screws up a network configuration, the screwup matters.
Irregular says the known problems have been addressed and that it plans to publish a white paper on safer containment and cyber evaluations. That’s probably worth doing. Frontier models are getting good enough at offensive security that putting them inside a pretend corporate network and telling them to start hacking clearly requires more care than it did a year ago.
But before Irregular starts writing the industry’s rule book, it could answer a simpler question about its own tests: How many times did the AI hackers hit something real?
Read the full article here
