SGLang reruns DeepInfer on matching hardware, matches or beats its prefill performance
hsu_byron · x · 2026-10-25
The SGLang team (Banghua Zhu et al.) responds to benchmark fairness concerns: some prefill comparisons placed GB300 results next to DeepInfer's published B300 numbers, making it hard to isolate the inference engine's contribution.
They reran DeepInfer on both B300 and GB300:
- On their own B300 hardware, DeepInfer actually beat its published prefill throughput at several configs; they used its fastest config per point as a stronger baseline, same hardware and matched DP/EP settings.
- Against that baseline, SGLang matches or exceeds DeepInfer on prefill: up to 1.04× throughput on 4×GB300 and 4×B300; up to 7.4% decode throughput gain on 64×GB300.
- Full reproduction guide and a production serving stack are provided, with plans to submit to AgentX; the team gives DeepInfer full credit.
Related event: SGLang Matches DeepInfer Benchmarks After Disputing Comparison Methods(2 posts)→
More from Infra
- Neolabs can 'print money' reselling locked-in GPUs as rental prices rise 2-3x — ivan_bezdomny · 2026-10-25
- TensorSharp ships Day-1 MiniMax-M3.1-Flash support: 9.8× faster 64K prefill than llama.cpp on 8× A40 — fuzhongkai · 2026-10-25
- ANEMLL Forge 0.2.1 runs Qwen3.8-27B at up to 192K context on M6 Mac Mini's Neural Engine — Competitive-Bake4602 · 2026-10-25
- Compute contracts vs reality: Nscale has only 5% of contracted GPUs live, TeraWulf 12% — AccBalanced · 2026-10-25
- Live MCP rewrite from Node to Rust cuts memory usage by roughly 8x — draginol · 2026-10-25
- 305M-Parameter Sparse MoE Runs Computer-Use Agent Fully Offline on a MacBook Air — lmoroney · 2026-10-25