LLMs aren't discrete: tokens are just ~0.00001% of the continuous process
Sauers_ · x · 2026-11-11
Responding to claims that LLMs are fundamentally discontinuous because they emit discrete tokens rather than a smooth continuous flow like humans, Sauers argues this is a misreading: tokens are intermediate readouts of an underlying continuous process. By his estimate, tokens carry only 0.00001% of the past information available to the model — the sampled output shouldn't be confused with the process itself.
More from Research
- Biswas Foundation gave $1.7M to 37 AI medicine projects; Cycle 2 opens Nov 15 — anshulkundaje · 2026-11-11
- Demo: training models to read K/V vector breadcrumbs left by other instances — Sauers_ · 2026-11-11
- EEG can't match implanted BCIs: physics caps scalp signals at ~70Hz — EricMTrautmann · 2026-11-11
- 2B model takes 1.7 days and 22 steps to land one FunSearch hit — cephaloform · 2026-11-11
- Baseten's All-Gather research conference unveils agenda with Noam Brown, Oriol Vinyals, Mark Chen — vjkaruna · 2026-11-11
- The 1960 Doomsday Equation That Predicts Infinite Population on Nov 13, 2026 — csvoss · 2026-11-11