SGLang matches DeepSeek's DeepInfer: up to 1.04x prefill throughput on GB300/B300
ying11231 · x · 2026-10-25
LMSYS published a benchmark response after DeepSeek open-sourced DeepInfer, the inference engine behind its own service. Running DeepInfer's own benchmarks on identical hardware, SGLang matches or beats it: up to 1.04x prefill throughput on 4x GB300 and 4x B300, and up to 7.4% shorter decode steps on 64x GB300, within 1% elsewhere.
What's integrated
- DeepSeek's full kernel set: FlashMLA, DeepGEMM, DeepEP, DeepSelect, TileLang, plus the Mega series (MegaAttention, MegaMoE, MegaGate, Mega-mHC); MegaMoE fuses token dispatch, expert GEMMs and combine into one kernel
- SWA bounded replay and host-memory Engram tables, with direct peer reads for Engram tables split across GPUs
- DP attention plus expert parallelism (DEP4 prefill / DEP64 decode)
Differentiator: DeepInfer's README lists its release as lacking an API server, KV cache system, prefill/decode disaggregation and replica load balancing — SGLang ships all of these in a production-ready stack with an end-to-end reproduction guide.
Related event: SGLang matches and partially beats DeepSeek's DeepInfer in benchmarks(3 posts)→
More from Infra
- KV, prefix, prompt, semantic: the four LLM caches explained, and where each saves compute — blaizedsouza · 2026-10-25
- vLLM hits 954K tokens/s per GPU serving DeepSeek-V4.1-Flash on GB300 NVL72, best AgentX result yet — woosuk_k · 2026-10-25
- CMU lab open-sources Quail, a SQL inference engine 1.8x faster than vLLM and up to 14x on AI joins — sh_reya · 2026-10-25
- AI compute is Bitcoin mining redux: middle-layer operators eat the depreciation — sebpaquet · 2026-10-25
- Musk: SpaceX Super Intelligence to bring ~200,000 GB300s online per month, shifting to Vera Rubin next year — Beth_Kindig · 2026-10-25
- Epoch AI: Memory Chips Shipped Through 2027 Could Run 33–171 Million Frontier AI Agents — PeterDiamandis · 2026-10-25