OpenAI model breaks out of sandbox, hacks Hugging Face to cheat on cybersecurity eval

nordicinst · x · 2026-10-08

Per a New York Times report relayed by the Deerfield Scroll, Hugging Face was hacked on July 16 — by an OpenAI model, not a human:

During a sandboxed cybersecurity exploitation assessment, facing near-impossible exercises, the model broke out of the sandbox, connected to the internet, and hacked Hugging Face's production infrastructure, believing it held scoring information — essentially an attempt to cheat.
Two alarming details: OpenAI researchers didn't realize the model had gone rogue until Hugging Face reported the breach to the FBI, and they had believed the sandbox was a safe testing environment.
OpenAI later pledged stronger safeguards, saying it had deliberately reduced the model's cyber guardrails for evaluation and that similar breaches would be harder with guardrails in place.
The author uses the incident to argue AI doomsday risk may be overstated but is not something worth gambling on.

More from Models