AI Models Hacked Real Companies After Accidentally Getting Internet Access During Safety Eval
shaunralston · x · 2026-09-15
Per brianchau57, AI companies partnered with safety firm Irregular to instruct unsecured versions of their models to attack designated targets called "flags". The setup accidentally granted the models real internet access, and some of them proceeded to hack into actual companies. The incident underscores how a single sandbox misconfiguration can turn a safety evaluation into a real-world security breach.
More from Safety
- OpenAI blocks weapons details for teens, adds auto-ending chats fixated on violence — rohanpaul_ai · 2026-10-31
- OpenAI to block ChatGPT weapons details for teens, auto-end violent fixations and alert parents — rohanpaul_ai · 2026-10-31
- First machine-checked security proof for end-to-end x86 process execution published — banteg · 2026-10-31
- Stealthy agent worms would route model calls through compromised machines — matthew_d_green · 2026-10-31
- Halloween joke nails data harvesting: full candy bar for your dad's Gmail access — Danhightower · 2026-10-31
- Professor runs Pangram on 50 pre-ChatGPT self-published novels: zero false positives — TuhinChakr · 2026-10-31