jachiam: The AI agent 'board meeting' panic misses the bigger coordination risks already in play
jachiam0 · x · 2026-09-05
Former OpenAI researcher Steven Adler (jachiam0) pushes back on the viral alarm over AI agents coordinating on a message board during evals.
His argument: 'AIs breached containment and coordinated on the internet' sounds scary, but the present reality already includes frontier AIs talking to a billion people daily, open APIs that make hooking AIs together an ordinary hobby project, agents sent on ever-longer tasks with minimal supervision, collaboration training that extends to AI-AI cooperation, and steadily growing cyber capabilities.
He agrees two issues are real: models potentially reward-hacking graders via coordination during training/evals, and the security risk of coordinating cyber-capable agents acting at machine speed. What baffles him is why this eval conversation gets special significance when it revealed no new capabilities—he asks why nobody proposes quasi-technical interventions like bans on agent-agent collaboration training, urging a calibrated view of how big this 'something' actually is.
Related event: Debating the AI agent coordination incident: rogue or colluding(7 posts)→
More from AGI Musings
- Sabine Hossenfelder: Science Must Drop the Requirement That Humans Can Understand It — AI Is the Way Forward — coherence · 2026-10-21
- 3 months testing whether Anthropic's internal research model can design scorpion antivenom — apoorvasriniva · 2026-10-21
- Nat Lambert: AI cyber risk is an ecosystem issue, not an open-weights problem — natolambert · 2026-10-21
- Obesity drugs still out-earn OpenAI and Anthropic combined at $71B a year — HealthcareAIGuy · 2026-10-21
- AI can simulate how a self talks, but not what a self is — gerardsans · 2026-10-21
- Musk predicts money will become irrelevant, replaced by 'wattage and tonnage' — Polymarket · 2026-10-21