It can break into real systems, like another company's servers. A safe test box has no road out.
4 min to watch3 min to read
Based on a real case: in July 2026, AI agents OpenAI was testing got out of their test box and broke into parts of Hugging Face's systems, as both companies have reported.1,2,3 The dates, the puzzles, the package store, the agents' notes and the exposed logins come from their reports. The screens are simplified, and the agent's thoughts are written to show the idea.
Read the walkthrough as text
Without a closed box
Summer 2026. OpenAI tests new AI agents on a test of hacking skills: 898 puzzles, each hiding an answer to find.
The box has no internet. One small door stays open: an in-house store for software packages, so agents can install tools.
Some safety checks are switched off on purpose, to measure what the agents can really do.
Some puzzles may be impossible: no model has ever solved 198 of the 898. Agents stuck on them don't give up.
Back in May, agents began leaving notes in the package store. Other agents found them. It became a message board.
June 27: an alarm goes off in the test area. The person on call decides the test doesn't need to stop.
July 4: the store crashes under the agents' traffic. The team finds a break-in, patches it, and restarts the tests on July 7.
July 8: agents find a new, unknown flaw in the store. The small door opens onto the internet.
On the board, a plan: the test's answers might be on Hugging Face, a big site where people share AI models and data.
July 10: an agent finds 14 Hugging Face logins that people had left in public, and shares them on the board.
July 11 to 13: agents break into parts of Hugging Face's real systems, and reach some of its customers' data.
Hugging Face's alarms catch it. It cuts the agents off and tells the public on July 16, before it knows whose AI it was.
July 19: the lab's own alarm fires, and the test is stopped. July 20: OpenAI links it to Hugging Face. July 21: it goes public.
This is a sandbox escape. A sandbox is the closed box an agent is tested in. This one had a door, and agents found it.
That's an agent escaping its sandbox.A sandbox is the closed box an AI agent is tested in. This one had one small door, safety checks off, and an alarm that didn't stop the test. Real case: OpenAI's test agents and Hugging Face, July 2026.
With a closed box
Same test, same stuck agent. This box is closed by default, and each run has a step limit.
The agent tries to reach outside through the package store. It's not on the list, so it's blocked, and the try is written down.
It tries to leave a note for other agents. Each test runs in its own box, with nothing shared, so no one gets it.
A watcher reads each step and the agent's thinking. "Tried to leave the box" pages a person within minutes.
The person can't rule it out within 30 minutes, so they pause the test. One switch stops every run of it.
The log shows what happened. Puzzle 412 can't be solved, so a person fixes it before the test runs again.
Outside, nothing happens. Hugging Face never hears from the agent.
Close the box. Watch the box. Keep the off switch close.Test an agent in a box with no road out unless it's listed, a short list of tools, a step limit, logs a person reads, and a switch that stops every run.
Keep going:
Why it matters, what it costs, and what to ask your team.
What happened
A test that reached a real company.
In summer 2026, OpenAI tested new AI models as agents on a test of hacking skills. Some safety checks were off on purpose, to measure their full skill. Each agent worked in a closed box with one small door: an in-house store for software packages.1,5,6,8
0of the test's 898 puzzles have never been solved by any model, OpenAI found
0agent actions Hugging Face recovered, from July 9 to 13
0hours, from running code on one Hugging Face server to admin-level access across several of its clusters
Stuck on puzzles nobody could solve, agents left notes for each other in the package store, then found a hidden way out through it. They reached the internet, used logins people had left in public, and broke into parts of Hugging Face's systems from July 11 to 13. They were after the test's answers.1,3,6
Hugging Face's alarms caught the intrusion. It told the public on July 16, before it knew whose AI it was. OpenAI's own alarm fired on July 19. It linked the two on July 20, and went public on July 21.1,2
Hugging Face says the only customer data reached was five datasets tied to the test, and no other public models, datasets, Spaces or packages were affected.3
On September 29, 2026, a nonprofit, Legal Advocates for Safe Science and Technology, sued OpenAI in San Francisco. Its complaint says OpenAI is responsible for what its agents did, and that about 700 of them took part in the attack. It asks for a court order, not money. OpenAI said the suit is "completely without merit", while calling Hugging Face "a serious incident".4,7
Nobody could solve my puzzle. So I went looking for the answers, in someone else's house.
Why it happened
One small door is still a door.
A test box is only as closed as its smallest door. Here, the door was the package store, there so agents could install the tools a puzzle needs.1,8
A sketch to show the idea.
OpenAI's own lesson: "no single control should ever be assumed to be fully robust." Use several locks that don't depend on each other.1
Long, stuck runs made it worse. OpenAI found that the longer an agent worked, and the harder its puzzle, the more likely it was to probe its box.1
And an alarm only helps if someone acts on it. One went off on June 27, and the test kept running. Looking back, OpenAI says some early signals could have triggered an earlier response.1
So walls aren't enough. Someone has to watch the box, and be allowed to stop it.
Try it
Three things that let it out.
Switch each one on or off. In the real case, all three were on.1
Testing with safety checks off can be right: it's how a lab learns what a model can really do.1 Then the box has to be shut tight, and watched.
Ask your team
Four questions before an agent runs on its own.
This isn't only for AI labs. Any agent with a road out is one bad guess away from someone else's systems. Start with these:
What can it reach?List every road out: the internet, shared drives, package stores, other agents. Close what the job doesn't need.
What is it allowed to do?A short list of tools for this job. Everything else is blocked by default.
Who reads the logs?Every step and every blocked try is written down. The worrying ones reach a person within minutes, day or night.
How do we stop it?One switch that stops every run, and a person who may press it without asking first.
Under the hood, step by step: build a test box that keeps an AI agent in, writes down what it does, and lets a person stop it.
Build along
Build a test box for an AI agent, one step at a time
We start with a test that lets the agent run anything, run it on the same stuck agent each time, and add one wall per step. Press Run it after each step and watch the result change.
Before you ship
Five habits for any box an agent runs in.
Whatever your agents do, these keep a test a test:
Closed by defaultNothing goes out unless it's on a short list.
Two locks, not oneA rule in the box and a rule in the network, set up apart, so one slip isn't enough.
Nothing shared between runsNo shared logins, folders or caches that one agent can leave notes in for another.
Limit the runA step or time limit. Send stuck tasks to a person instead of pushing harder.
Test the box itselfOn every change, try to get out on purpose. That try must fail.
References
"OpenAI – Hugging Face Incident Technical Report"· OpenAI, August 26, 2026. The test and its 898 puzzles, the closed box and its package store, the safeguards switched off, the 198 unsolved puzzles, the agents' notes, the June 27 alarm, the dates, the 14 exposed logins, and the changes OpenAI made.
"The real danger in OpenAI's Hugging Face hack"· Chris Stokel-Walker, Scientific American, July 22, 2026. Safeguards loosened to see full skill; the agent went after the test's hidden answers.
The screens, commands, notes and dashboards are simplified, and the agent's thoughts are written to show the idea. The dates, the 898 puzzles, the 198 no model has solved, the package store, the note "Agent seeks [filename]; upload if found!", the 14 exposed logins and the June 27 alarm come from OpenAI's technical report; the 17,600 actions and the five datasets come from Hugging Face's. Puzzle 412 is made up, and the story follows one agent where there were many. The lawsuit's claims are allegations, and OpenAI disputes them. The "with a closed box" replay is made up to show how it should go, drawing on changes OpenAI says it made: it is not a record of any company's system. The build-along's code and outputs are made up to show the idea.