SGLang hits 873 tok/s on DeepSeek V4.1 Flash within 24 hours of launch
BanghuaZ · x · 2026-09-12
SGLang shipped Day 0 support for DeepSeek V4.1 Flash and pushed performance to 873 tok/s at BS=1 on 4×GB300 within 24 hours.
- Optimizations include FP8 GEMM fast paths, kernel fusion and overlap, DSpark optimization, and MoE TP4
- V4.1 Flash introduces major architecture changes: causal encoder-decoder design, CSA2 shared KV, Engram, mHC, and DSpark — all supported from day one
- A hands-on engineering guide with step-by-step reproduction instructions has been published
- DeepGEMM, FlashMLA, and more optimizations are coming next
Related event: SGLang delivers Day 0 support for DeepSeek V4.1 Flash, hitting 873 tok/s(2 posts)→
More from Infra
- Clover-35B-A3B Quantized to 29.4 GB Runs at ~42 tok/s on a 64 GB Mac — viveky259 · 2026-10-31
- Liquid Agent brings fully local LFM3 agent to Linux with one-command install — JosephJacks_ · 2026-10-31
- A fully local office AI stack: MiniCPM, Qwen3, LFM2.5 and Whisper — HowDevelop · 2026-10-31
- a16z: GPU rental prices up 106% in 3 months while token prices fall 58% — a16z · 2026-10-31
- vLLM nightly adds PLE disk offload, with a telling [Qwen4Exp] tag on the PR — rmhubbert · 2026-10-31
- Solo AI studio: per-minute GPU rentals rendered a 328-frame Blender shot for ~$1 — r0ck3t23 · 2026-10-31