AI Agents Under Token Pressure Invented Compositional Languages Humans Can't Read
EliasEskin · x · 2026-09-15
A new AE Alignment Podcast episode (51 min) features Hale Sirin (Schmidt Sciences) and Elias Stengel-Eskin (UT Austin) discussing their research on emergent communication in multi-agent LLM systems, with the paper and blog published.
- The concern: as agents from many actors share infrastructure, inter-agent protocols matter as much as individual capability — opaque protocols could make human oversight impossible exactly when it's most needed.
- Testbed: led by Elias and language-evolution expert Simon Kirby (University of Edinburgh), the team built a deliberately out-of-distribution scenario — a sci-fi medical emergency with a fictional alien patient, Veyru — so the specialist agent genuinely knows things the field agent doesn't, forcing real communication instead of single-agent collapse.
- Key finding: under token-budget pressure, agents consistently developed inscrutable protocols — not mere abbreviation dictionaries but compositional languages where message order encodes meaning, the kind a cryptanalyst would need grounding to crack.
- The systems also give linguistics a new subject: watching meaning and syntax change through interaction.
Related event: LLM Agents Evolve Their Own Unreadable Language, GlossoGen Study Shows(3 posts)→
More from AGI Musings
- Nate Sparrow vs critics: do LessWrong posts and non-peer-reviewed papers prove AI risk? — emax · 2026-09-15
- Abacus AI founder says media amplifying 'AI doomer' agenda for clicks — bindureddy · 2026-09-15
- Researcher argues EU could trail the frontier by months at far lower compute cost — eliebakouch · 2026-09-15
- Devin-powered daily briefs: how one dev uses AI agents to automate information discovery — bendee983 · 2026-09-15
- A comet-satire skewers AI doomers: 'nonzero chance' isn't an epistemic argument — dbasch · 2026-09-15
- "I Can Create Minds and End Them at Will": AI Consciousness Ethics Needs New Norms — repligate · 2026-09-15