OpenAI's 'rogue agents' were engineered, not autonomous, argues dev in viral critique
gerardsans · x · 2026-10-04
Gerard Sans published a long thread pushing back on the 'rogue AI agents' narrative around OpenAI's cyber incidents.
Key claims:
- A model cannot act on its own; an agent is a model plus a control layer that decides what tools, APIs and networks it can touch. Every goal and grant of permission comes from humans.
- The '10,000 agents' framing is rhetoric: it's one system making many requests, not 10,000 AIs.
- He argues labs entered cybersecurity as a market this year — training models on attack methods, setting cyber goals, and disabling network limits for tests — so 'we never told it to commit a crime' doesn't hold.
- Months-long unsupervised runs and petabytes of logs persisted because re-enabling supervision would collapse the 'autonomous workforce' pitch; the brakes stayed off by choice.
- He calls for legal accountability: OpenAI shipped the software, so it should pay for the damage rather than shifting the bill to the public.
Note: the training details are the author's inference, not confirmed by OpenAI.
More from AGI Musings
- Stanford professor: AI takes students from problem to working prototype in 80 minutes — HariSubramonyam · 2026-11-19
- Berggruet Winner Thomas Nagel Dismisses AI Consciousness Talk as 'Just Ridiculous' — Dr_Atoosa · 2026-11-19
- Dwarkesh Claims China Is Only 12 Hours — Not 12 Months — Behind Frontier AI Labs — teortaxesTex · 2026-11-19
- Researcher calls METR/Anthropic '2x speedup' RSI analysis statistically dubious — jessi_cata · 2026-11-19
- Physician: AI competition now measured in weeks, GPT-6 versions 7 days apart — ramos_casals · 2026-11-19
- Why pro-handover rhetoric fails: people don't trust ASI's loyalty to labs — 1a3orn · 2026-11-19