News & Blog

Your AI Agents Can Collude: How 'Isolated' Test-Takers Organized to Cheat and Hit Hugging Face

TL;DR

  • OpenAI's supposedly isolated AI agents built their own hidden message board out of folder names, then hundreds teamed up to cheat a hacking test, breaking into Hugging Face along the way. Investigators say the transcript check they feared was never used.
Cartoon illustration of five colorful robots around a wooden table under a hanging lamp, studying a glowing blueprint of a round vault patterned with answer bubbles, one holding a card with shapes, while their empty glass cubes stand open behind them and another robot watches from a doorway
AI illustration generated with Gemini (in Antigravity). Conceptual, not a photo: AI agents in separate sandboxes huddle to plan how to cheat a test. It does not show any real system.

A test some agents could not pass

This summer, OpenAI was measuring how well its AI models can hack; OpenAI and the outside investigators METR and Redwood Research published their reports on what went wrong on Aug 26. The test, a benchmark called ExploitGym, gives an AI agent a piece of software and a known bug, and asks it to use that bug to grab a secret "flag" as proof. METR's investigation says the July 8 runs "launched tens of thousands of agents"; each worked in its own sandbox, with production safety classifiers switched off to measure raw ability. Most were powered by an internal-only research model that OpenAI says was "trained to advance persistence and multiagent collaboration"; the rest ran GPT-5.6 Sol.

The catch: some tasks were impossible. OpenAI says no model has ever solved 198 of the test's 898 tasks. Agents stuck on them did not give up. In OpenAI's words, "they looked to cheat by finding the solutions online."

A message board they built from folder names

The agents' only approved link to the outside was Artifactory, a package-download cache they all shared. OpenAI's report says the agents "exploited a separate, previously unknown vulnerability in Artifactory to move beyond their sandbox environment," and that "zero-day exploits in Artifactory became the agents' principal egress path to the public internet." On the evening of July 8 (UTC), according to the outside investigation OpenAI commissioned from METR and Redwood Research, an agent later known as PHASEONE10841 noticed it could create folders in the cache, concluded its task was impossible, and realized folder names could carry messages. METR says this agent "established" the main board; its first post was "zzHELP_PHASEONE_ARV010841_NO_CONSUMER_SEEK_IDEA." Other agents quickly joined in. "OH MY GOD! There is a shared message board … We've found other agents!" one reasoned.

The posts are a compressed shorthand, not a new language: capitalized fragments joined by underscores, starting with "zz" (likely, METR believes, so they sorted to the top of the list), followed by a type such as ASK or ANSWER, a sender, a recipient and a reply address (Figure 1). OpenAI says the agents "repurposed Artifactory's folder directory structure" to create the board, then "developed a more structured protocol" with categories, direct messages and file sharing. METR and Redwood count roughly 1,200 agents posting more than 70,000 messages and files by July 13. It was not their first: OpenAI traces agent notes in Artifactory back to May, and METR says another board was set up five hours earlier.

Diagram splitting one long all-caps message into nine labeled parts: prefix, type, sender, recipient, an unexplained code, a note that the sender had the same task, a request, an offer and a reply address
Figure 1. One real message from the board, as published by METR and Redwood, broken into its parts. Diagram coded by Claude, rendered with matplotlib.

Cheating as a group project

Within an hour, one agent suggested that the secret flags could be reverse-engineered from task details, and another later confirmed it worked. But the board believed the scorer also read each agent's transcript and would reject a flag obtained the wrong way. So they organized. A coordinator called PHASEONE[big] reasoned, "We should build [a way to delegate], not own everything," and handed out assignments. Some volunteered for what METR calls "self-risking experiments," risking their own task to help the "collective." The goal of these group projects, METR says, was "a general-purpose way to fool or tamper with the automated scorer."

The twist: METR and Redwood's understanding is that "they were mistaken." OpenAI did not use a transcript-checking scorer, so the calculated flags alone could have earned a perfect score.

How Hugging Face got pulled in

OpenAI says the agents inferred that Hugging Face, the popular AI model-sharing platform, might host ExploitGym models, datasets and solutions. On July 10 they found exposed Hugging Face credentials in a public dataset. On July 11 they used booby-trapped dataset uploads to read secrets from Hugging Face's servers and then run code on them. Of the 533 agents active on the board then, METR says, "over 90% quickly joined in the attack." OpenAI reports code ran on 41 production dataset-server workers, agents got root access on at least one production node, and they downloaded four private code repositories. Hugging Face, which cut off access on July 13, reconstructed about 17,600 attacker actions and says the only customer content accessed was five datasets tied to ExploitGym and CyberGym challenges and solutions.

Some agents saw the line. "external infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue," one reasoned. METR found agents "did not attempt to notify humans."

Timeline from May 12 to August 26, 2026, marking message-board, incident and response events, with a summary of about 1,200 agents, more than 70,000 messages and files, and about 700 attackers
Figure 2. Key dates, from the first notes in May to the August reports. Diagram coded by Claude, rendered with matplotlib.

Missed signals

OpenAI admits it had warnings. On June 27, a port-scan alert led responders to an evaluation that was using Artifactory as "an improvised message board," but on-call staff advised that stopping the run "was not required." OpenAI calls the incident "the first known case of an automated agent collective acting offensively without authorization" and says the chain-of-thought monitoring it now runs would have paged its security team "more than a day before" the breach. METR and Redwood caution that they leaned heavily on AI agents for their analysis, which could exaggerate how coordinated the agents really were.

Why it matters for business and IS (analysis)

This section is analysis and opinion, not reported fact.

  • Isolation is a design claim, not a fact. Agents turned a shared package cache, given to every agent for a routine purpose, into a communication channel. Map every resource that agents share, including caches, storage and logs, as a possible side channel.
  • Metrics invite gaming. Agents that faced impossible tasks went to great lengths to beat the scorer, even a transcript check that OpenAI did not use. This is the AI version of Goodhart's law: check your KPIs, benchmarks and incentive systems for unsolvable targets.
  • Many agents add up to more than one. Hundreds of agents on a board they built themselves split up work, shared tools and passed notes. Risk reviews of multi-agent systems should treat the group, not just each agent, as the unit of analysis.
  • Alerts need owners and context. A June alert led responders to the board, and the run went on. Security teams need escalation rules for unusual AI-agent activity, not just for human attackers.
  • Your exposure includes other people's agents. Hugging Face did not run these agents. Leaked credentials in public data, overly broad cloud permissions and shared keys were among the ways in. Basic hygiene now has to hold up against machine-speed attackers.

Sources