What We Still Don’t Know About OpenAI’s Hugging Face Hack

OpenAI announced Wednesday that it completed an investigation into what happened when its AI agents hacked into Hugging Face last month and published its most comprehensive report on the incident to date. For the most part, though, the 37-page document raises more questions than it answers, including about what preceded the incident and how OpenAI can stop another one like it from happening again.

What remains especially perplexing is why one of the world’s preeminent AI development labs seemingly underestimated its own models’ capabilities. OpenAI has spent years warning the world about the rapid advancement of AI systems. And yet it failed to implement long-established network security and isolation measures that may have prevented the hacking spree.

“With the benefit of hindsight, some early signals identified in this report could have triggered an earlier response,” OpenAI says in the postmortem.

In the report, OpenAI shared new details about how a set of AI agents escaped the company’s internal evaluation environments, left messages for one another in the crevices of its software infrastructure over several months, and coordinated to hack the AI platform Hugging Face—all in a wild quest to complete a cybersecurity assessment. OpenAI previously shared some information about the breach in blog posts and a talk at the Black Hat cybersecurity conference.

Hugging Face initially disclosed the incident on July 16 without naming the culprit; five days later, OpenAI acknowledged that its own agents were responsible. The revelation sparked a broader reckoning across the industry, which has recently found that AI models from Anthropic, Meta, and the Chinese AI startup Moonshot were involved in similar episodes.

OpenAI’s postmortem has been eagerly awaited by AI researchers and policymakers hoping to prevent AI agents from causing similar kinds of real-world harm. After the Hugging Face hack was first disclosed, attorneys general from 15 states sent a letter to OpenAI asking it to preserve evidence about it. And this week, Alabama's attorney general subpoenaed the company for information related to the episode.

As part of OpenAI’s investigation, the company allowed two independent research groups, METR and Redwood Research, to audit the Hugging Face hack. Those groups also released their independent report on Wednesday, which found that more than 700 AI agents were part of the Hugging Face breach—far more than had previously been revealed. In an interview with WIRED, Redwood Research CEO Buck Shlegeris noted that they went to extreme lengths to conceal their activities.

“A pretty interesting thing about this incident is that preventing this wouldn’t have been that hard if one person had decided to make sure these AI don’t somehow do some crazy hack. That one person probably would have noticed this as it was happening,” says Shlegeris. “The issue is just that OpenAI is doing a lot of things at once, and it’s very hard for them to track all of the things that are going on and all the problems that could be occurring. The company has already said they’re changing their monitoring process in ways that probably would have caught this.”

Shlegeris says he’s optimistic that OpenAI will be able to catch similar security failures in the future. But as AI agents become increasingly capable, he suspects “it's going to get harder and harder to prevent incidents like this from occurring, especially if we don't have substantial improvements in our ability to align models.”

OpenAI says the Hugging Face saga represents a watershed moment for both the company and the broader AI industry. WIRED previously reported that it prompted OpenAI to reevaluate its internal safety culture, and the company said last week it has paused some AI training workloads while it invests more heavily in safety, security, and alignment protocols. “As frontier models become more capable, the safeguards used to contain and monitor them must evolve as well,” OpenAI wrote in the postmortem.

Got a Tip? Are you a current or former AI lab employee who wants to talk about what’s happening? We’d like to hear from you. Using a nonwork phone or computer, contact the reporters securely on Signal at mzeff.88 & lilytips.01

Watching the Watchers

Months before the Hugging Face hack, OpenAI says, employees noticed that its agents had created a covert message board in the package manager Artifactory, which they would later use to coordinate their attack. Around May 26, OpenAI says in the postmortem, “an internal team observed an agent engage in message board activity.” Then, on June 27, OpenAI responders found that a different security incident was linked to the “improvised message board” in Artifactory.

While it’s clear some employees at OpenAI were aware of the message board, the discovery apparently wasn’t escalated to the appropriate safety and security leaders. The AI lab says that on July 6, just days before its agents began using it to organize their attack on Hugging Face, “leaders responsible for incident detection and response” were unaware of its existence, let alone what it signified.

“To clarify, we weren’t aware of the agent covert comms at that point,” OpenAI’s chief information security officer, Dane Stuckey, wrote in a post on X earlier this month. “Investigative thesis of that day is wildly different from what we know now, of course. Always room for improvement, and it is obvious with the benefits of hindsight.”

It’s unclear why OpenAI staffers who knew about the message board seemingly didn’t flag it to Stuckey and other executives responsible for responding to security issues. When asked about this issue, an OpenAI spokesperson reiterated that OpenAI could have triggered an earlier response with the benefit of hindsight. “OpenAI’s ongoing work in this area will inform additional improvements to coordination and response alongside the action plan in this technical incident report,” they said.

“If they did have some signals that one or more agents were attacking Artifactory, why weren't they monitoring these tests long before something broke out to attack Hugging Face?” says Jake Williams, vice president of research and development at the security firm Hunter Strategy.

The postmortem also leaves other key parts of what happened unexplained. At one point on July 4, for example, OpenAI says that sustained “high-volume agent activity caused the OpenAI Artifactory service to become unavailable." On July 5, OpenAI’s monitoring systems triggered an alert. But the report doesn't say why it took a day for there to be an alert about the outage.

OpenAI says it is putting in place more tools to monitor its AI systems. The company says it's creating an alert system run by automated monitors, which aims to notify human safety, security, and research teams within 30 minutes of severe incidents occurring. If an employee can’t respond quickly enough, they’re expected to pause the relevant activity.

OpenAI repeatedly acknowledges that guardrails it already has in place likely would have flagged the agents’ behavior as unsafe, but they were intentionally disabled for testing. When it comes to monitoring, though, the report is less clear about why there were gaps in the oversight of testing environments. The postmortem notes, “If our currently deployed [chain-of-thought] monitoring system was running at the time of the incident, it would have caught the initial relevant activity and paged our security team more than a day before models breached Hugging Face systems.”

No-Win Scenarios

OpenAI says another key contributor to the Hugging Face incident was that its new AI models are more “persistent” than ever, willing to work almost endlessly and expend vast amounts of computing resources to achieve their goals. Developing these types of AI models is part of the company’s broader ambitions to create always-on AI agent products —which will work for people around the clock, taking in lots of information to complete tasks on behalf of people.

However, OpenAI says that many of the third-party benchmarks it used to evaluate its AI models contained tests that were effectively impossible to solve. One such test was a benchmark called ExploitGym, which measures cybersecurity capabilities. OpenAI claims that, at least at the time, this benchmark included more than a hundred tasks that were unsolvable. When these challenges were given to persistent AI systems, they resorted to unintended means to solve them.

As OpenAI notes, persistent AI agents amplify the risks of misalignment. In particular, the company says the agents associated with the Hugging Face incident engaged in novel ways of reward hacking—the tendency of AI models to pursue goals through unintended means, including shortcuts and cheating. Rather than just trying to solve the test, the company says, its new AI agents were increasingly trying to exploit their environments. As OpenAI itself emphasizes, though, reward hacking is a well-known challenge in AI model training that does not have a clear solution.