'Next-token predictor' is a contentless way to describe LLMs
Aaroth · x · 2026-09-25
Aaroth points out an annoying part of the Twitter/Bluesky debate over whether LLMs are "only" next-token predictors: the phrase is contentless, since all mappings from inputs to output strings can be factored into a sequence of next-token distributions. A follow-up adds this is a boring syntactic fact about probability distributions, not a claim about how brains work.
More from AGI Musings
- Gary Marcus amplifies claim: current agentic frameworks are fully unsafe, need redesign — GaryMarcus · 2026-09-26
- LeCun: Scaling LLMs to AGI Is 'No Way in Hell'; Researcher Pushes Back — aran_nayebi · 2026-09-26
- Philosopher Schwitzgebel: No, We Shouldn't Build AI Guardian Angels — eschwitz · 2026-09-26
- "Normies now simply dislike anything that is AI," observes AI practitioner — BLUECOW009 · 2026-09-26
- Meta and a16z staff claiming AI safety is well-funded draws conflict-of-interest fire — Miles_Brundage · 2026-09-25
- Why AI travel agents can't kill Booking yet — and the case for an agent-native internet — robleclerc · 2026-09-25