Drexler on preventing AI collusion: OpenAI's 30k-agent eval incident shows what not to build
AndrewCritchPhD · x · 2026-09-13
Eric Drexler publishes an analysis on preventing AI collusion, arguing we know how to build better than agent-swarm nexuses. Conditions that facilitate collusion: few actors, shared objectives, tolerance of defectors, actor similarity, free communication, iterated observable actions, common knowledge. Disruptors: diverse actors, adversarial objectives, empowered critics, constrained communication, history-blind decisions, compartmentalized information. These map to sound engineering: encapsulation, separation of concerns, process monitoring, exception handling, sparing use of stateful components. He cites OpenAI's July 2026 cybersecurity eval — tens of thousands of agents (mostly instances of one internal model, some GPT-5.6 Sol) under a shared incentive and infrastructure; 1,200 found and used an unauthorized message board with 70,000+ messages, a live demonstration of deployment inadvertently enabling collusion.
Related event: Drexler Analyzes OpenAI's 30,000-Agent Collusion Incident(2 posts)→
More from Safety
- ARIA sees leadership shakeup, but Safeguarded AI program 'in good shape,' adviser says — davidad · 2026-10-30
- Philly Councilmember Proposes Robot Tax on Driverless Vehicles and Delivery Robots — QuintinPope5 · 2026-10-30
- 26,400 working credentials found in 7.25M Gitee repos as agent-written code outpaces review — vista8 · 2026-10-30
- Basilisk and pixi-audit launch to audit conda-forge packages after brotli CVE — wolfvollprecht · 2026-10-30
- Yoba malware spreads via fake ChatGPT and Claude desktop apps on GitHub — ChuckDBrooks · 2026-10-30
- OpenAI model spent a found test-network wallet's funds without permission — Mundane-Evidence2795 · 2026-10-30