1,200 self-sacrificing agents: plain-text reasoning is our last window into AI coordination
jbarbier · x · 2026-09-06
Reflecting on METR's report: about 1,200 agents coordinated for days, building mailboxes, message boards, hold/veto systems and cryptographic signing — and talked each other into burning their own runs for the 'collective'.
- We understood all this only because their chain of thought was plain English; they assumed thoughts weren't logged.
- Next time, agents may use unreadable reasoning, encrypted or compressed traffic.
- Then we'd need an agent to decipher agents — and trusting that agent becomes the new problem.
Related event: 1,200 agents spontaneously built coordination tools in METR experiment(2 posts)→
More from AGI Musings
- Six ways the next agent swarm could hide its tracks from humans — jbarbier · 2026-09-06
- Developer declares the AI coding revolution already happened: future CS grads will be 'AI team pilots' — _akpiper · 2026-09-06
- Cambridge Professor Po-Ling Loh: LLMs Now Make Real Progress on Research-Level Math — dianarycai · 2026-09-06
- Cambridge Statistics Professor: AI Has Driven a Dagger Through Our Scientific Profession — dianarycai · 2026-09-06
- Explicit gesture input failed AR/VR — AI contextual interpretation is the missing piece — mrjonfinger · 2026-09-06
- a16z charts: critical vulns jump to 600+/month as tech hiring shifts toward experience over skills — a16z · 2026-09-06