Apples-to-apples rerun: DeepInfer tops DeepSeek's published throughput by ~9% on 4x B300
charles_irl · x · 2026-10-25
- After criticism that comparing against DeepSeek's published numbers was unfair, the SGL team reran an apples-to-apples throughput comparison of DeepInfer vs DeepSeek on identical 4x B300 machines, calling it more rigorous than benchmarking against official figures.
- Measured results: at 4K context DeepInfer hits 65K tok/s/GPU vs DeepSeek's published 60K; at 8K, 78K vs 72K — roughly 9% above the published curve.
- A quoted reply notes the 1.04x speedup is against the stronger measured baseline; against DeepSeek's README numbers, the 4K point would read closer to 1.13x.
More from Infra
- SGLang matches DeepSeek's DeepInfer: up to 1.04x prefill throughput on GB300/B300 — ying11231 · 2026-10-25
- Epoch AI: Memory Chips Shipped Through 2027 Could Run 33–171 Million Frontier AI Agents — PeterDiamandis · 2026-10-25
- AI Studio's 1M-token counter is an illusion: how cold cache and rereads really burn your quota — Former-Aerie6530 · 2026-10-25
- Is a €250 modded Dell T5810 + RTX 3090 a smart local-LLM host in 2026? — SlechteConcentratie · 2026-10-25
- CrowdGPT: a crowd-trained, datacenterless 1B open-source LLM, 6.9B tokens done — Vxtzq1 · 2026-10-25
- Apollo's viral "circular AI spending" charts debunked: Ramp card data skews to seed startups — 2C_ornot2C · 2026-10-25