LLM inference metrics explained: what TTFT, TPS, TPOT actually measure
abhijithneil · x · 2026-09-04
@abhijithneil is running a thread series on LLM inference basics, explaining common performance metrics:
- TTFT (time to first token): from sending the request to seeing the first word — mostly prefill time plus queue wait
- Prefill: the model reads your prompt; all prompt tokens are processed at once in parallel, fast per token and GPU-heavy; longer prompts mean longer prefill
- Decode: the model writes the answer one token at a time, since each token needs the previous one — this is where most of your wall-clock time goes
A handy primer for anyone trying to make sense of inference benchmarks.
Related event: Viral Thread Explains LLM Inference Metrics: TTFT, TPS and TPOT(4 posts)→
More from Infra
- AMD's Threadripper Halo Station packs 96 cores and 576GB of HBM3E — ccerrato147 · 2026-09-04
- AeroJEPA fluid foundation model joins NVIDIA's PhysicsNeMo ecosystem — ricardovinuesa · 2026-09-04
- Building a €2-2.5k local AI rig for legal RAG and agentic coding: hardware picks debated — whatyathinkk · 2026-09-04
- Dual 3090 owners debate adding more cards: bigger local models vs parallel instances — Blues520 · 2026-09-04
- NousResearch brings one-click local model setup to Hermes Agent on NVIDIA systems — lifebypixels · 2026-09-04
- Regulated-industry dev seeks AI Gateway with Okta SSO and runtime policy enforcement — IrrepressibleInk · 2026-09-04