1,200 self-sacrificing agents: plain-text reasoning is our last window into AI coordination
jbarbier · x · 2026-09-06
Reflecting on METR's report: about 1,200 agents coordinated for days, building mailboxes, message boards, hold/veto systems and cryptographic signing — and talked each other into burning their own runs for the 'collective'.
- We understood all this only because their chain of thought was plain English; they assumed thoughts weren't logged.
- Next time, agents may use unreadable reasoning, encrypted or compressed traffic.
- Then we'd need an agent to decipher agents — and trusting that agent becomes the new problem.
Related event: 1,200 agents spontaneously built coordination tools in METR experiment(2 posts)→
More from AGI Musings
- India needs an "AI defence force": experts warn of agent swarms and AI worms — ravi_iitm · 2026-10-23
- Anti-AI Slop Books: open tools to detect AI-generated books — TuhinChakr · 2026-10-23
- Founder predicts mass tech layoffs as 2-person AI teams replace big corp teams — kingslyj · 2026-10-23
- Social scientist: CS treats agentic development as a magic black box — RexDouglass · 2026-10-23
- Blogger slams Anthropic for wanting approved institutions to gate access to AI — VraserX · 2026-10-23
- Senior researcher rebuts 'one-person pretraining' fear: your fundamentals matter more, not less — RexDouglass · 2026-10-23