How LLMs learn: guess the next word trillions of times, everything agentic is a wrapper
sethjuarez · x · 2026-09-08
Seth Juarez (Microsoft AI Platform evangelist) published an 18-minute-read article, "How the Machine Guesses the Next Word," opening a series aimed at building up to agents.
Key points:
- An LLM does exactly one thing: given text, compute the most likely next token. When it answers "Paris" to "The capital of France is," it doesn't "know" anything — it just picked the highest-probability token.
- Text is first chopped into tokens (via schemes like byte-pair encoding); the model's entire world is a sequence of integer token IDs, never raw letters.
- Training in one breath: feed it the internet, hide the next word, guess, check how wrong, nudge less wrong — repeated trillions of times. "Grammar and reasoning just fall out the side."
- Thesis of the series: every agentic capability is a runtime wrapped around this single humble mechanic.
More from coding & agent
- Open-source local studio bundles image, video, music and AI movie maker offline — lelus78 · 2026-10-26
- LangChain and Tavily to co-host NYC event on production agents using live web data, Nov 12 — LangChain · 2026-10-26
- SentinelForge: evidence-driven AI software repair with human-in-the-loop verification — Byte_Forge_2811 · 2026-10-26
- Booking bug lesson: never let the model name the service, let code own IDs — HivraAi · 2026-10-26
- Dev hacks reMarkable e-ink tablet to let Opus draw and animate — Stefania_druga · 2026-10-26
- Godot games are trivially decompilable — compile C# to native DLL for real protection — mflux · 2026-10-26