DeepSeek's controlled inference benchmarks praised as the most compelling roofline curves yet
teortaxesTex · x · 2026-10-22
ekzhang1 praised DeepSeek's benchmarking methodology for controlling all free variables, calling its roofline curve the most compelling recent one and arguing every inference engine should benchmark this way, with KV caches and prefill/decode measured separately.
The attached chart shows DeepSeek V4.1 Flash decode throughput vs. interactivity under fixed conditions (EP64DP64, 384+64 experts, 131k context, nextn=3, 75% per-token acceptance): 26,000 tok/s/GPU at batch 256 (100 tok/s per user) down to 2,500 tok/s at batch 8 (320 tok/s per user). A quoted tweet jokes that inferenceX deprecating 8k/1k configs caused the controversy.
More from Infra
- Seroter's daily list: OpenAI models land on Google Cloud, PHP CVEs, Genkit Go 1.14 — rseroter · 2026-10-22
- inference.net tops OpenRouter third-party hosts at 14% share, serving ~10% of daily volume — Ronangmi · 2026-10-22
- Eden Deploys Vercel's Eve Agent to Cloudflare With One Command, Fully Open Source — ritakozlov · 2026-10-22
- Node.js ships emergency fix for high-severity Brotli decoder RCE in v22/24/26 — cramforce · 2026-10-22
- PyTorch's TorchTitan adds DeepSeek-V4, Kimi K3 support, RL training and big scale gains — PyTorch · 2026-10-22
- Epoch AI tracks 95 AI data centers with 14.2 GW — still just 44% of global AI compute — haider1 · 2026-10-22