LLM decoding explained: prefill reads in parallel, decode writes token by token
abhijithneil · x · 2026-09-04
Parts 5-6 of @abhijithneil's LLM inference primer explain the two core stages:
- Prefill: the model reads your prompt; all prompt tokens go through at once in parallel — fast per token and works the GPU hard; long prompts mean long prefill
- Decode: the model writes the answer one token at a time, because each token needs the previous one to exist first; this is where most of your wall-clock time goes
Foundational concepts for understanding inference latency and throughput metrics.
Related event: Viral Thread Explains LLM Inference Metrics: TTFT, TPS and TPOT(4 posts)→
More from Infra
- AMD's Threadripper Halo Station packs 96 cores and 576GB of HBM3E — ccerrato147 · 2026-09-04
- AeroJEPA fluid foundation model joins NVIDIA's PhysicsNeMo ecosystem — ricardovinuesa · 2026-09-04
- Building a €2-2.5k local AI rig for legal RAG and agentic coding: hardware picks debated — whatyathinkk · 2026-09-04
- Dual 3090 owners debate adding more cards: bigger local models vs parallel instances — Blues520 · 2026-09-04
- NousResearch brings one-click local model setup to Hermes Agent on NVIDIA systems — lifebypixels · 2026-09-04
- Regulated-industry dev seeks AI Gateway with Okta SSO and runtime policy enforcement — IrrepressibleInk · 2026-09-04