vLLM serves DeepSeek-V4.1-Flash at 1.3B tokens per dollar, 830K tok/s per GPU
TheZachMueller · x · 2026-10-17
The vLLM team congratulated DeepSeek on V4.1-Flash-1017, noting that better scores plus shorter outputs make it a great fit for long agent loops, with per-step speed being the serving side's job.
Key numbers on the SemiAnalysis AgentX benchmark:
- 1.3B tokens per dollar of TCO
- 830K tok/s per GPU (cached + input + output) at 100 tok/s per user
- 5x agentic throughput improvement since the project began
vLLM teases the next round of optimizations, and observers note the efficiency gap is brutal for inference providers not yet on the latest hardware.
Related event: vLLM Boosts DeepSeek-V4.1-Flash Throughput 5.3x in Three Weeks(2 posts)→
More from Infra
- SonicBoom v0.0.3: compiler-first AI runtime beats llama.cpp decode by 1.18x, calls for GPU benchmarking — nalaginrut · 2026-10-19
- M5 Ultra Mac Studio rumored with 512GB RAM; 6 units chained could run 2T-parameter models locally for ~$100k — SumitGup · 2026-10-19
- Apple Silicon's memory economics: M5 Ultra offers 5.6x better $/GB/s than DGX Spark for LLM inference — SumitGup · 2026-10-19
- Mac Studio M5 Ultra vs Dual DGX Spark: Long-Input Inference Benchmarks Show Split Verdict — GabGarrett · 2026-10-19
- DLite: History-Free Speculative Decoding Fixes 80% Acceptance Drop in Agentic AI — SJTU · 2026-10-19
- Gartner warns $3.5T AI infra spend at impairment risk by 2030; its own stock is down 61% — firstadopter · 2026-10-19