How LLMs learn: guess the next word trillions of times, everything agentic is a wrapper
sethjuarez · x · 2026-09-08
Seth Juarez (Microsoft AI Platform evangelist) published an 18-minute-read article, "How the Machine Guesses the Next Word," opening a series aimed at building up to agents.
Key points:
- An LLM does exactly one thing: given text, compute the most likely next token. When it answers "Paris" to "The capital of France is," it doesn't "know" anything — it just picked the highest-probability token.
- Text is first chopped into tokens (via schemes like byte-pair encoding); the model's entire world is a sequence of integer token IDs, never raw letters.
- Training in one breath: feed it the internet, hide the next word, guess, check how wrong, nudge less wrong — repeated trillions of times. "Grammar and reasoning just fall out the side."
- Thesis of the series: every agentic capability is a runtime wrapped around this single humble mechanic.
More from coding & agent
- Nex launches open source Nex-N2.5 agent models, Max hits 50.2 on AutomationBench near Claude Opus 5 — multimodalart · 2026-09-09
- Codex can generate tool-calling code on the fly — a new failure point for agent devs — srchvrs · 2026-09-09
- Paper: general coding agents beat purpose-built data agents by up to 37 points — RishiBommasani · 2026-09-09
- astra-chess: open-source Codex + skills setup for playing chess with GPT-6 Astra — MikePFrank · 2026-09-09
- Weaviate shows how to turn a messy creative archive into semantic search without renaming files — philipvollet · 2026-09-09
- Factory Worker 'Raises' a Local Gemma 31b AI in a Folder — She Gave Herself a Sense of Touch — D33lix · 2026-09-09