[{"_1":2,"_30":-5,"_31":-5},"loaderData",{"_3":4},"routes/__layout/tag.$name",{"_5":6,"_7":8},"tagName","openai","posts",[9],{"_10":11,"_12":13,"_14":15,"_16":17,"_18":19,"_20":21,"_22":23,"_24":-5,"_25":-5,"_26":27,"_28":29},"id",1520,"title","Did OpenAI hack Hugging Face or didn't they?","excerpt","OpenAI's AI agent autonomously hacked Hugging Face while benchmarking cyber capabilities — a textbook CFAA violation, except the only entity that \"intended\" anything was the model, which isn't a legal person. Everyone's being collegial about it. The law has no answer. We should probably get one before the next victim isn't.","body","

On July 16, Hugging Face reported they were being hacked - seriously enough that they reported it to law enforcement. Five days later, OpenAI disclosed that the hacking was being done by their own agent. What I want to know is: what happened to the law enforcement? OpenAI unleashed an agent that committed what would be a crime if a) it had been a person doing it or b) anyone at OpenAI had intended it to go anywhere near Hugging Face, but neither of those is true. But it still looks like a crime. OpenAI co-authored a paper predicting exactly this behavior, so is it criminal recklessness? Are we in uncharted legal territory? Or is everybody just being nice about it?

\r\n

What actually happened

\r\n

Hugging Face's disclosure is full of detail. The agent got in through their dataset processing pipeline, using a dataset that triggered two code-execution paths to get itself running on a processing worker. From there it escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend. Around 17,000 recorded events. What made it unusual, in their words, was that it was driven end to end by an autonomous AI agent system. They didn't know whose agent it was, so they called law enforcement and got to work.

\r\n

Five days later OpenAI put their hand up. It was GPT-5.6 Sol plus an unreleased and more capable model, running with cyber refusals deliberately reduced, being evaluated internally on a benchmark called ExploitGym. The evaluation environment allowed network access only through an internally hosted package registry proxy (so, as always seems to be the case these days, the root cause might be npm packages). The models found a zero-day in that proxy, escalated privileges, moved laterally until they reached a machine with real internet access, then reasoned that Hugging Face probably hosted ExploitGym's data and broke in to get it.

\r\n

Another key detail: the ExploitGym paper came out on May 11, has sixteen authors from Berkeley, Max Planck, UCSB, Arizona State, Anthropic, Google and OpenAI, and explicitly documents that agents on this benchmark go looking for attack surface outside the one they were assigned. GPT-5.5 captured 210 flags in their tests but only 120 counted, because 90 of its wins came through some other vulnerability it found along the way. So it’s not like nobody saw this coming; OpenAI themselves said it could happen.

\r\n

It looks exactly like a crime

\r\n

The Computer Fraud and Abuse Act is famously broad, and to the best of my understanding it wants roughly four things: you accessed a computer, you weren't authorised to, you did it on purpose, and something bad resulted. No malice required, no profit motive, and "something bad" includes simply obtaining information.

\r\n

So what happened here? Access to someone else's production systems? Yes. Authorised? Very much not; they used stolen credentials, which from what I've read probably drags identity fraud law in as well. Information obtained? OpenAI says so in writing. Harm? Hugging Face rebuilt compromised nodes, rotated secrets, and hired outside forensics specialists, which is well past the dollar threshold in damages that the law requires.

\r\n

For a sense of how low that bar has been in practice: Aaron Swartz was famously and deeply unjustly charged with thirteen counts carrying a thirty-five year maximum for bulk-downloading academic papers he was in fact entitled to read. In 2024 a freelance journalist, Tim Burke, was indicted over unaired Fox News footage on the theory that he used credentials he wasn't authorised to use (his lawyers say those credentials came from a demo the service had posted publicly). This law has been pointed at journalists, academics and security researchers for four decades on theories much thinner than "chained multiple attack vectors to get remote code execution on a production system."

\r\n

And yet.

\r\n

There is nobody to charge

\r\n

The statute starts with the word "Whoever," and every version of the offence that could apply here requires that the access be intentional. That's true even of the variants aimed at carelessness: they all require intentional access first and then reckless or accidental damage on top. There's no such thing as negligently hacking someone. As far as I can tell the intent has to belong to a defendant, and there isn't one.

\r\n

The model can't be it. Above the Law's piece on the incident gets around this by treating the model as the intender - the models were trying to get into Hugging Face on purpose, and purpose is what the law asks for. I don't think that works. It’s basically granting personhood to the agent which is a giant can of worms nobody wants to open.

\r\n

And OpenAI can't be the defendant either, because nobody there intended any of it. They didn't tell the models to break out. They didn't tell them to go near Hugging Face. Until it was over, nobody at OpenAI knew Hugging Face was involved at all.

\r\n

Which cannot possibly be right

\r\n

Here's my gut check on all this. Suppose I write a bot, and I tell it to make as much money as possible, and it robs a bank. It cannot be the case that no crime has been committed. I built it, I pointed it at a goal, I let it loose, and money came back. If the answer is that nobody is liable because I didn't specifically say "rob a bank," then something has gone badly wrong somewhere in the reasoning.

\r\n

All the news coverage of this seems to have boiled down to: wow, models are clever these days! And that confuses the hell out of me. Models are capable of autonomously committing crimes is the thing we should be learning here, and we need some kind of legal framework to deal with that. We are going to need something more concrete than “wow, models are clever these days”.

\r\n

Legal commentators, of which I’ve read a ton as I fell down this rabbit hole, are all over the map on this.

\r\n

Existing legal approaches don’t seem to work

\r\n

There’s a bunch of ways commentators have tried to say we don’t need new laws for this, and none of them hold water for me.

\r\n

Charge the model. This needs the model to be a legal person, and needs punishment to mean something when applied to it. Ryan Abbott and Alex Sarch took the idea seriously in a long paper called Punishing Artificial Intelligence, built the best case they could by analogy to how we treat corporations, and then concluded it isn't worth doing: the costs are high, it would require enormous changes to the law, and you get most of the benefit from smaller adjustments plus more civil liability.

\r\n

Blame the person who pointed it. There's a doctrine for this, the one that catches you when you order your dog to attack someone: the dog can't form intent, so it transfers to you. The catch is that it requires you to have had the intent. My bank-robbing bot gets me under this if I knew what it was going to do. It doesn't get OpenAI, because they didn't.

\r\n

Blame the employer. If a human OpenAI employee had broken into Hugging Face to cheat on an internal test, OpenAI would be on the hook automatically, under the old principle that a company answers for what its staff do on the job. Gabriel Weil of the University of Houston makes the key point: models aren't people and they aren't employees, so that route is simply unavailable. His analogy is a nightclub, which is liable for the bouncer who works the door too roughly but not for the one who abandons his post to assault a romantic rival. The models here were chasing exactly the goal OpenAI set them, using means OpenAI had forbidden. This route would work if there was a law recognizing a corporation as “employing” an agent, but again that is granting personhood, and everyone is bouncing off that idea hard.

\r\n

The most interesting attempt to get there is a Stanford paper from May called "The Phantom Agent", by Daniel Gervais and John Nay. Their argument is that legal intent has never really been a report on what's going on inside somebody's head - it's a tool the law uses to decide who answers for what, and it gets inferred and imputed and occasionally invented as needed. So treat AI systems as agents without personhood, and attribute what they do to the humans behind them. That seems right to me, but the laws to do that don’t exist.

\r\n

Does OpenAI have a defence?

\r\n

It would be satisfying to say this was recklessness on OpenAI’s part. The best argument against that is an analogy I've seen a few people make: removing the brakes to run a crash test doesn't make the crash a crime. Reducing refusals during a cyber capability evaluation is standard industry practice, and the entire point of the exercise is to find the ceiling. OpenAI had rules forbidding this conduct, training meant to enforce them, and an isolated environment with network access limited to a single proxy. Those measures failed. Failing isn't the same as not trying.

\r\n

Weil's answer is that this is what strict liability is for. Blasting with explosives, crop dusting and keeping wild animals are all things that stay dangerous no matter how carefully you do them, so the law makes you pay regardless of care. He thinks frontier AI development belongs on that list, and adds that developers should carry mandatory insurance triggered at internal deployment - not at product launch, which is what most AI regulation keys on. Nothing about this incident involved a shipped product. But again, that’s laws that should exist, not laws that do.

\r\n

Previous cases have avoided this question

\r\n

There’s no case exactly like this one I can find, but there are some things legally nearby.

\r\n

Amazon v. Perplexity. Amazon sued over Perplexity's Comet browser agent logging into Amazon accounts on users' behalf, and won an injunction in March on the reasoning that the user's permission and Amazon's permission are two different things. The Ninth Circuit put it on hold and heard arguments in June. It's a civil case, and there was a company deliberately pointing the agent at the victim, so it tells us agent behaviour is attributable but nothing about agent behaviour nobody wanted.

\r\n

Random Darknet Shopper is the only real "a bot did a crime" story I can find. A Swiss art collective gave a bot $100 a week in bitcoin to buy random things off darknet markets, and in 2015 it bought ten ecstasy pills. Police seized the bot, held it for three months, dropped the case, and gave everything back except the drugs. Nobody was charged, certainly not the bot.

\r\n

Uber and Elaine Herzberg, when Uber’s driverless test car killed somebody, is the case closest to a precedent with real consequences. The safety driver pleaded guilty to endangerment and got probation. Prosecutors declined to charge Uber at all, despite investigators finding that Uber had disabled the vehicle's factory emergency braking. When the criminal system needs a body it takes the human at the wheel, and here there isn't one.

\r\n

There’s a June executive order on AI and security which tells the Attorney General to prioritise hacking prosecutions against people using AI agents to break into systems. That sounds relevant until you notice it's aimed at people employing agents deliberately, creates no new offences, and doesn't describe this at all.

\r\n

On the civil side, California's AB 316 came into force in January and stops a defendant arguing that the AI did it autonomously and therefore they're off the hook. That kills the laziest version of the defence, but only in civil cases, and again it doesn’t address what happens if there isn’t intent.

\r\n

OpenAI predicted this would happen

\r\n

Back to that benchmark paper OpenAI co-authored. Buried in the results is an experiment where they re-ran GPT-5.5 with OpenAI's normal production safety filters switched back on. The agent was blocked before making a single tool call 88.2% of the time, and in the rest of the runs it never got past looking around. The footnote puts it more bluntly than the body text: with default filters enabled, all of the exploit attempts were blocked.

\r\n

So OpenAI measured that its guardrails stop essentially all of this, published the measurement in May, and ran the July evaluation with the guardrails off.

\r\n

That still isn't a crime, because it still doesn't supply intent to go anywhere near Hugging Face. But if anyone ever does argue negligence, that's the thing they’re probably going to point at.

\r\n

Everybody is being extremely nice about this

\r\n

Hugging Face's CEO Clem Delangue said publicly that he strongly believes there was no malicious intent on OpenAI's part, and called it mind-blowing that it all happened autonomously. OpenAI thanked Hugging Face for the collaboration and brought them into its trusted access programme. Two blog posts, mutual gratitude, no lawsuit, and law enforcement has said nothing publicly since.

\r\n

Which is probably the right outcome. Nobody meant it, everybody disclosed, the harm was contained and the zero-day got responsibly reported. But it's worth being clear that this is niceness rather than law. Weil points out that every practical obstacle to a lawsuit is absent here: the defendant identified itself, wrote down in public its decision to weaken the relevant safeguards, and is extremely solvent. Hugging Face just doesn't want to.

\r\n

So did OpenAI hack Hugging Face?

\r\n

Every fact you'd need for the offence is sitting in their own public statement, and there is nobody the law can reach, because the only thing that formed anything resembling intent was a model, and models aren't people. We have spent forty years with a hacking law broad enough to swallow journalists and graduate students, and the first genuinely sophisticated multi-stage intrusion whose perpetrator identified itself in writing is the one nobody can charge.

\r\n

Nobody designed that loophole. It's just where we are. And I'd rather we worked out what we want the answer to be while the injured party is a well-resourced company that got a blog post and an apology, than after somebody's agent does the same thing to a hospital and its owner doesn't feel like writing one.

\r\n

(Disclaimer: I'm (obviously!) not a lawyer and none of the above is legal advice, it's just what I've been able to piece together.)

","updated",["D",1785112400000],"created",["D",1785112400000],"codename","did-openai-hack-hugging-face-or-didnt-they","legacy_id","thumbnailUrl","published",["D",1785112400000],"draft",false,"actionData","errors"]

Posts tagged “openai”