What concepts can't words describe? Stanford team frames it as channel coding
stanfordnlp · x · 2026-10-30
A research thread shared by Stanford NLP poses a deeper question: what concepts can and cannot be described with words?
The authors formalize this as a communication channel problem and a double scaling law problem: if a concept is described (1) as well as provably possible and (2) with optimal language models, can it be transmitted between two LLMs in words?
And does that limit shift as models scale — or are some constructs permanently stuck beyond verbal description? A theoretical exploration of the expressive limits of language and LLMs.
More from AGI Musings
- Crisp, clear writing becomes a huge moat when AI does all the writing — Saboo_Shubham_ · 2026-10-30
- Dan Wang: London is already a post-AGI city — 'not functional, but fun' — herbiebradley · 2026-10-30
- Philosopher argues AI could usher in a golden age of mathematics—if we redefine mathematical progress — Dr_Atoosa · 2026-10-30
- A 4-person team ships 77,000 changes/month with AI agents — how devs stay relevant — yoobinray · 2026-10-30
- Ben Horowitz: continual learning is the next big tech revolution and value accrues to its builders — brianryhuang · 2026-10-30
- Anthropic API Key Leaked; Agents Detected and Rotated It Within 15 Minutes — tylerjdunn · 2026-10-30