Signal65 MLPerf analysis: Azure GB300 tops DeepSeek-R1 interactive serving at 260k tokens/sec
ryanshrout · x · 2026-11-10
NVIDIA's Ryan Shrout highlights Signal65's analysis of audited MLPerf Inference v6.1 DeepSeek-R1 results, showing over 260k tokens/sec on validated GPU clusters. The Interactive scenario enforces a 1.5s time-to-first-token and 15ms per output token, matching conversational/coding workloads. Signal65 argues reasoning models only pay off when latency stays low, and each extra GPU to hold response time raises serving costs; among same-generation, same-count GPU systems, Azure ND GB300 v6 led the Interactive scenario.
More from Infra
- Google ships four TPU systems in one generation, adds confidential inference and max-context variants — AccBalanced · 2026-11-10
- The math behind Anthropic's 5-minute and 1-hour prompt cache options — bigaiguy · 2026-11-10
- Why Send This to the Cloud? Resume Screening Runs Locally on an RTX Spark — draginol · 2026-11-10
- Copy-from-document speculative decoding speeds AI extraction 4.7x (up to 19x) vs naive vLLM — sh_reya · 2026-11-10
- Compute is the first crux of AI takeoff: substitution estimates diverge wildly — austinc3301 · 2026-11-10
- CoreWeave Launches Claude Code Skill to Build GPU Clusters and Serve Models from the CLI — altryne · 2026-11-10