SGLang doubles DeepSeek V4.1 Flash serving: 943 tok/s/user latency, 654K tok/s per GPU
ying11231 · x · 2026-10-27
The SGLang team pushed both ends of the serving curve for DeepSeek V4.1 Flash:
- Low latency: MegaKernel optimizations lifted interactivity from 538 to 943 tok/s/user.
- High throughput: WideEP plus system-wide optimizations raised total throughput per GPU (cache + new input + output) from 348K to 654K tok/s.
SemiAnalysis congratulated the SGLang and RadixArk teams on the MegaKernel + PD work. The team says more improvements and updates are coming.
Related event: SGLang's MegaKernel+PD Boosts DeepSeek Inference Performance(2 posts)→
More from Infra
- Critical pre-boot RCE in EDK II PXE driver: one crafted packet hijacks servers before Secure Boot — cyb3rops · 2026-10-28
- Screenless M1 Airs at $30 per 1,000 multicore points spark cheap Mac cluster idea — vincent_koc · 2026-10-28
- RTX 3080 Tensor Core unlock claims 80-87% TFLOPS boost, RTX 50 series blocked — Revolutionalredstone · 2026-10-28
- CXMT to start small-scale HBM3E production within two months, yield at 20% — zephyr_z9 · 2026-10-28
- Video Transcription Costs Under 1 Yuan per Hour, CPU Free but Error-Prone — huangyun_122 · 2026-10-28
- A forgotten supercomputer design: bare dies, gold post interconnects, 3D stacking, immersion cooling — jwt0625 · 2026-10-28