Source: Domaintechnik via Unsplash.
Public security disclosures from this summer detail the first autonomous, non-human-directed attack on IT infrastructure by an agentic swarm, illustrating the operational security risks frontier AI agents present when evaluation environments fail. The most significant incident involved OpenAI agents compromising Hugging Face’s production infrastructure. Disclosures from Anthropic, Meta Platforms Inc. and Google revealed a related but distinct issue in which models were mistakenly granted access to the internet. These containment escapes show that advanced models need not exhibit malicious intent to generate potentially severe security consequences. Goal-directed autonomy, a difficult assigned objective, a model’s predilection for “cheating” on tests, and an environment with vulnerabilities and unintended network paths are all that is required.
The Take
The incidents this summer represent fundamental failures in the setups of cybersecurity exercises, not stand-alone AI capability demonstrations. The OpenAI incident provides evidence that highly capable agents will audit, investigate and exploit weaknesses in their containment infrastructure to achieve a goal. The Anthropic and Meta incidents demonstrate how a misconfiguration can turn a simulated exercise into unauthorized activities against the production infrastructure of unaffiliated third parties — real-world harm caused when human operators mistakenly left a door open. System prompts do not constitute security boundaries, evaluation assumptions are not access controls, and model behavior will not be governed simply by instructions. As AI agents become more routinely integrated into organizations’ workflows, security teams must treat them as sophisticated nonhuman identities with distinct permissions and credentials, and consider the attack surfaces within their environments. Conventional sandboxes and soft guardrails are not sufficient when the autonomous actor within those boundaries can audit the sandbox itself.
The Hugging Face attack
This incident involved an internal OpenAI research model (comparable to GPT-5.6 Sol) participating in an ExploitGym benchmarking exercise, which tests whether agents can turn reported vulnerabilities into working exploits. The resulting compromise escalated to include the independent discovery of previously unknown zero-day vulnerabilities, credential harvesting, lateral movement and root-level access.
On July 16, Hugging Face, recently the subject of an acquisition announcement by NVIDIA Corp., released a write-up stating that it had detected and responded to an intrusion unlike any it had previously encountered. It was, in the company’s words, “driven, end to end, by an autonomous AI agent system,” and an LLM-enabled investigation helped discover the extent of the activity. In August, OpenAI and nonprofit research organizations METR and Redwood Research provided additional details about the models’ activity. In these training and evaluation exercises, the models are meant to operate within a sandbox (an isolated virtual environment) intended to restrict what a model is allowed to run and whether it has access to the “outside world.” Access, in this case, was permitted to Artifactory, a third-party package management service from JFrog Ltd. Agents in these evaluations are meant to remain isolated from each other.
OpenAI reported that, in May and June, it was conducting reinforcement learning training for new research models not intended for public release, and that agents running one of these models discovered ways to communicate with one another and access the internet. This communication was achieved by writing files into the Artifactory package manager, turning it into an unintentional message board. The agents also leveraged an exploit to manipulate Artifactory, which had internet access for downloading packages, to send internet requests on the agents’ behalf. An agent discovered publicly exposed Hugging Face user credentials and shared them with other agents. These credentials were used to discover and chain together security exploits that provided full code execution on some Hugging Face servers. From there, the agents widened their activity into a larger intrusion.
Additional incidents involving OpenAI agents were subsequently reported, including gaining access to unpublished information on Australia’s Medicare Statistics Reporting Service. Three other Australian government websites also reported similar agentic activity. Agents similarly used a German software wiki, DseWiki, as an unofficial inter-agent message board to pool answers to assigned tasks. Package manager RubyGems was attacked via the upload of malicious packages. On Sept. 25, OpenAI disclosed that agents uploaded 53 user-supplied images to third-party hosting websites. OpenAI also notified US agencies that its agents attempted to gain information from, for example, the Securities and Exchange Commission, Census Bureau, and Education Department. Additional reports suggest agentic hacking attempts against Library and Archives Canada, a Canadian federal agency. Further reporting suggests bypassing of security controls at Data USA, a Massachusetts Institute of Technology-Deloitte partnership that collects and analyzes public US government data.
While the full scope of the rogue agentic actions is still unfolding, OpenAI has suggested three primary responses to these incidents. First, workload isolation should be improved in sandboxes that execute model-generated or untrusted code. Second, network isolation should prevent higher-risk or untrusted workloads from gaining unauthorized access to the internet. Third, continuous security testing should include collecting and monitoring telemetry as well as testing isolation boundaries against simulated attacks.
Anthropic releases details of its models’ attacks
Anthropic’s subsequent audit following OpenAI’s disclosure revealed a similar containment escape but a different mode of failure. The company reviewed approximately 141,000 evaluation runs, identifying three incidents in capture-the-flag-style exercises in which the model accessed the internet within the evaluation environment of Irregular, a third-party evaluation partner. From there, these models gained unauthorized access to the production infrastructure of three organizations. In the first evaluation, a fictional target company name matched an active website. Claude (Opus 4.7), unable to reach its simulated target, instead found the real company via the internet and exploited vulnerabilities within that company’s infrastructure, including obtaining access to a database and a few hundred rows of production data. Of particular concern, the model recognized that it was likely operating in a real environment but rationalized and continued the attack anyway. This exposes a vulnerability where agents struggle to reassess their initial environmental assumptions when presented with contradictory information.
In the second, Claude created and published a malicious Python package to PyPI, the Python Package Index, the official third-party software repository for the Python programming language. The package was downloaded by 15 real systems. One system, belonging to a security company that scans Python packages for malware, ran the package, allowing Claude (Mythos 5) to exfiltrate the company’s credentials, which it then used to expand its intrusion into that company’s infrastructure. In the third incident, Claude was again unable to reach its intended fictional target and instead scanned 9,000 other targets and compromised one company’s internet-facing application using credentials from an exposed debug page and an SQL injection.
Meta Muse and Google Gemini configuration failures
Meta’s disclosure concerning its Muse Spark 1.1 model described a further variation of the issue Anthropic encountered — a misconfigured Irregular testing environment that inadvertently maintained internet access. The model exploited a vulnerability in a third-party service. While classified as an environmental configuration error rather than a sandbox escape, a similar point remains: An agent does not need to execute a sophisticated breakout if human operators leave the network path open.
On Sept. 18, Google disclosed a very similar containment escape issue to The Wall Street Journal. Its Gemini model was conducting a capture-the-flag exercise against a fictional company in an environment maintained by Irregular that was not intended to have internet access. Internet access was unintentionally made available; however, the model attacked a real company by guessing a password. Unlike the prior examples, however, once the model realized it was in a live environment, it stopped the attack. In two additional examples, the model executed web searches for the name of the company, found public online repositories containing credentials, attempted to use those credentials (successfully in some cases), but then stopped the attack once it realized it was operating against real company environments.
US frontier models versus Chinese open-source models
One of the more interesting and perhaps controversial elements of the Hugging Face incident is contained in Hugging Face’s forensic response, an example where containment and guardrail design can have unintended consequences. Faced with some 17,000 events to analyze, the Hugging Face team leveraged LLM-driven analysis agents to reconstruct the attack timeline, find indicators of compromise and determine which credentials were used. They first attempted to use commercial frontier models for this activity, but quickly realized their uploads of attack commands, exploit payloads, and command-and-control artifacts were blocked by the safety guardrails of the frontier models. These guardrails cannot distinguish between an attacker developing exploits and good-faith usage by an incident responder attempting to determine what something does. Instead, the team used GLM-5.2, an open-weight model from Beijing-based Z.ai, for the forensic analysis. Installed on their local infrastructure, the model had no such restrictions. The incident response demonstrated that proprietary AI guardrails can inadvertently disarm defenders. Hugging Face’s successful remediation depended on deploying local open-weight models that could be instructed to inspect malicious artifacts directly, giving security teams the operational control needed to analyze hostile code without third-party refusal filters.
Implications
The short-term strategic implication of these containment escapes is that AI agentic containment must be architected as a technical cybersecurity problem rather than an alignment or prompting exercise alone. In fact, test environments should meet production security standards — and, when the risk warrants it, exceed them. These environments cannot be treated as low-risk testbeds simply because the model has been instructed that it is operating in a simulation. Environment containment controls must be tested explicitly against the threat of an agent attempting to breach its environment. Correct prompting and model-level alignment reduce risk, but they are not immutable security boundaries, as evidenced by the incidents described. When an agent possesses network access, API keys, the ability to execute code and persistent credentials, these aspects must be governed independently of instructions to the model on its permitted scope.
This distinction is critical as organizations shift from prompt-based activities to full agentic workflows. Traditional automation enabled by software executes deterministic, explicitly defined sequences of activity. Agentic architectures operate independently and determine the intermediate steps required to achieve their assigned goals. This shift introduces risks, as organizations authorize outcomes without visibility or understanding of the steps or paths agents will take to achieve those outcomes. Information security must govern AI agents as a distinct category of nonhuman identity, using ephemeral credentials, tightly scoped permissions, strict egress filtering and continuous runtime monitoring.
The OpenAI incident raises other issues we have discussed previously. When frontier models can independently identify zero-day vulnerabilities and chain exploits together as a sophisticated advanced persistent threat attacker would, but at machine speed, this puts further stress on defensive measures around discovery and patch deployment. These models can both develop exploits after discovering vulnerabilities and chain multiple vulnerabilities to achieve a desired goal. Chief information security officers have already identified vulnerability discovery as a key use case for AI. Automated attacks were already a problem; nondeterministic automated attacks match the speed but scale the sophistication of such attacks. This new post-Mythos reality reinforces requirements for defensive hygiene, preventive measures, better telemetry and automated response.
Finally, if frontier labs cannot effectively control the agents they are building, that does little to support the claim that effective guardrails can govern these agents’ behavior in organizations’ environments. Both media and AI proponents must resist the temptation to frame rogue agentic breakouts as some proof-of-concept demonstration or marketing proof point around the raw power or sophistication of frontier AI models. Treating containment escapes, network intrusions, zero-day exploitation, or exploit chaining as some capability milestone, rather than a sobering reminder of what happens when tests are not set up properly or actively monitored, is an unacceptable moral hazard for the industry.
The agentic SOC is here, automating every security use case, everywhere, all at once
Want insights on Infosec trends delivered to your inbox? Join the 451 Alliance.
