Autoregression explained: a language model is just a next-token guesser in a loop
sethjuarez · x · 2026-09-04
Seth Juarez published an 18-minute primer on how LLMs actually work, setting up a series on building agents.
Key points:
- Thesis: a language model is nothing but a next-token guesser; every agentic capability is a runtime wrapped around that one mechanic.
- Text is first chopped into tokens (e.g. via byte-pair encoding); the model's entire world is integer token IDs.
- The autoregressive loop — guess a token, append it, feed it back, repeat until a "done" token — is the engine under every chatbot.
- Follow-up posts plan to build up, primitive by primitive, to agents that plan and act autonomously.
More from coding & agent
- PostgreSQL 19 slips to November as LLM-assisted code review floods committers with edge cases — sanjaykalra · 2026-10-21
- LangChain adds Human-in-the-Loop beta to Managed Deep Agents — LangChain · 2026-10-21
- Voxtral 3 ships a LiveKit-based voice agent SDK streaming listen-think-speak — MistralAI · 2026-10-21
- Open-sourced pixel art normals generator ships as Rust CLI and wasm library — HugoDuprez · 2026-10-20
- Eval researcher teases guide to subjective-task evals, with a curated resource list — cwolferesearch · 2026-10-20
- Eon cuts engineer onboarding from 2 days to 2 minutes with an MCP Gateway on Bedrock AgentCore — AWS ML Blog · 2026-10-20