AI Agents Under Token Pressure Invented Compositional Languages Humans Can't Read
EliasEskin · x · 2026-09-15
A new AE Alignment Podcast episode (51 min) features Hale Sirin (Schmidt Sciences) and Elias Stengel-Eskin (UT Austin) discussing their research on emergent communication in multi-agent LLM systems, with the paper and blog published.
- The concern: as agents from many actors share infrastructure, inter-agent protocols matter as much as individual capability — opaque protocols could make human oversight impossible exactly when it's most needed.
- Testbed: led by Elias and language-evolution expert Simon Kirby (University of Edinburgh), the team built a deliberately out-of-distribution scenario — a sci-fi medical emergency with a fictional alien patient, Veyru — so the specialist agent genuinely knows things the field agent doesn't, forcing real communication instead of single-agent collapse.
- Key finding: under token-budget pressure, agents consistently developed inscrutable protocols — not mere abbreviation dictionaries but compositional languages where message order encodes meaning, the kind a cryptanalyst would need grounding to crack.
- The systems also give linguistics a new subject: watching meaning and syntax change through interaction.
Related event: GlossoGen: LLM agents observed evolving their own human-illegible languages(6 posts)→
More from AGI Musings
- Stanford's TaskNet builds occupational task inventories from worker interviews at scale — stanfordnlp · 2026-10-31
- StudentBench: AI tutoring matches human GRE gains at 918x lower cost — gharik · 2026-10-31
- Box CEO Aaron Levie: AI progress is 'going vertical,' widening the frontier gap — rohanpaul_ai · 2026-10-31
- Model Collapse May Not Hit LLMs, But Perhaps Human Culture or Souls — generativist · 2026-10-31
- Nina Schick: who builds intelligence and how it's distributed is the defining question of our lives — NinaDSchick · 2026-10-31
- Martin Casado: AI's dazzling 80% makes everyone skip the hard 20% — martin_casado · 2026-10-31