vLLM dev: software optimization can yield superlinear gains, up to 80x in one generation
woosuk_k · x · 2026-11-17
Responding to abstract AI supply-vs-demand stock analyses, Woosuk Kwon (vLLM) argues real-world performance is nonlinear to FLOPS, HBM bandwidth, and model size. Within a single hardware generation, architecture and software optimizations matter enormously — e.g., DeepSeek v4.1 Flash can deliver 80x higher performance with the right optimizations, and superlinear gains are achievable.
More from Infra
- Minneapolis office-to-data-center conversion lifts value from $49M to $280M — DeGeneralDimes · 2026-11-17
- Dylan Patel: Nvidia Is 'Fucked' Without Jensen Huang; Musk's Data Centers Use Less Water Than Three In-N-Outs — elonmusk · 2026-11-17
- Ex-Apple Engineer Details Siri Speedups: Cold Starts Cut from 25% to 4% of Requests — jdluk87 · 2026-11-17
- NVIDIA DGX Spark HPC Developer Kit clusters four GB10 units with 128GB each — AIFlow_ML · 2026-11-17
- DeepSeek-V4.1-Flash hits 200 tok/s on Vera Rubin as agent ships Engram-table sharing fix — woosuk_k · 2026-11-17
- DeepSeek V4.1 Flash hits up to 93.3x per-chip throughput on Vera Rubin NVL72 vs GB300 — Xianbao_QIAN · 2026-11-17