antirez: for local LLM inference, fast prefill beats high t/s — 20-30 t/s is enough

teortaxesTex · x · 2026-09-29

antirez argues tokens-per-second is the wrong headline metric for local inference: the real game is shortening the thinking phase, and 20-30 t/s suffices with fast prefill — a hint Chinese open-model providers may act on. teortaxesTex counters that providers know this already; generating training data and training simply takes far more compute.

Related event: antirez: fast prefill and shorter thinking matter more than generation speed for local inference(3 posts)→

Original post →

More from Infra

Infra channel →