Next-token prediction doesn't imply lack of understanding; understanding resides in architecture
burny_tech · x · 2026-08-29
Countering the critique that LLMs can't understand because they just predict the next token, the author argues that prediction is merely the objective function, while understanding emerges from the architecture. A theoretically optimal predictor might simulate everything, similar to how humans use understanding for cloze tasks.
More from AGI Musings
- The alignment problem may be the business model: agreement is cheaper than truth — krishnan · 2026-08-29
- Opinion: once AGI arrives, the very expectation of privacy will evaporate — ShaneKaiGlenn · 2026-08-29
- AI Researcher Predicts Lunar Manufacturing Possible by 2035 — herbiebradley · 2026-08-29
- Physical world progress is too slow; we need to accelerate — Dr_Singularity · 2026-08-29
- AI may be removing the bottom rungs of the career ladder — rational-minority · 2026-08-29
- AI reliability improves 4-10x slower than capability, hindering automation — sayashk · 2026-08-29