DeepSeek 4.1 Flash hits 600K tokens/s on a single 8×B300 node, vLLM beats DeepInfer
kbesplv · reddit · 2026-10-21
warmctl benchmarked DeepSeek 4.1 Flash (FP4 experts, DEP8/DSpark/Engram) on a single 8×B300 node comparing vLLM 0.31 vs DeepInfer.
Key results:
- Over 600K total tokens/s throughput
- Output speed (1K→1K): vLLM 55K tok/s vs DeepInfer 27.9K
- Prefill on 32K prompts: vLLM 590K tok/s vs DeepInfer 65K
- Cold start to serving takes 930s; warmctl warm start cuts it to 88s (from 15 minutes)
vLLM wins across the board on this node, and warm-start tooling dramatically improves deployment turnaround.
More from Infra
- NVIDIA pivots to local AI: DGX Spark clusters to 8 units with 1TB unified memory — ryanshrout · 2026-10-21
- Daytona raises Series B to be the execution layer for AI agents; Meta ran Llama post-training on it — mattturck · 2026-10-21
- Anthropic cofounder: cut token costs 1000x and enterprises will just use 1000x more tokens — gabriel1 · 2026-10-21
- China shipped 1,730 crystal pullers to US in a quarter as Musk plots 20 GW buildout — rickasaurus · 2026-10-21
- Nvidia unveils plug-in DSX Pod data center at GTC Berlin to fix Europe's AI infrastructure lag — Wired AI · 2026-10-21
- Google Cloud now serves OpenAI, Anthropic, and Google frontier LLMs with hot-swapping — rseroter · 2026-10-21