Autoregression explained: a language model is just a next-token guesser in a loop

sethjuarez · x · 2026-09-04

Seth Juarez published an 18-minute primer on how LLMs actually work, setting up a series on building agents.

Key points:

Thesis: a language model is nothing but a next-token guesser; every agentic capability is a runtime wrapped around that one mechanic.
Text is first chopped into tokens (e.g. via byte-pair encoding); the model's entire world is integer token IDs.
The autoregressive loop — guess a token, append it, feed it back, repeat until a "done" token — is the engine under every chatbot.
Follow-up posts plan to build up, primitive by primitive, to agents that plan and act autonomously.

More from coding & agent