Baseten pushes DeepSeek V4.1 Flash to ~2,500 tok/s on Vera Rubin, 1.7x faster than GB300
ying11231 · x · 2026-11-17
Baseten shared new low-latency results for DeepSeek V4.1 Flash: 2,500 tok/s on Vera Rubin at batch size 1 with TP4, just 1.9ms per step — up to 1.7x faster than its previous GB300 numbers. The team is working with the SGLang project and radixark to upstream the optimizations into open source; details were presented by Tri Dao and colleagues at last week's SGLang summit.
Related event: Baseten Pushes DeepSeek V4.1 Flash to ~2500 tok/s on Vera Rubin(3 posts)→
More from Infra
- Unsloth Dynamic v3.1 UD-IQ1_M on RX 7900 XT: 53 tok/s with 192K context — zxspectrun · 2026-11-17
- Alchemy PR adds read-your-writes: clients auto-send command receipts with every read — samgoodwin89 · 2026-11-17
- Pipeshift claims 40% faster first token than Sarvam serving Gemma 4 31B at 150B tokens/day — NirantK · 2026-11-17
- LoopFill: training-free speculative decoding speeds up looped Transformers — UIUC-CS · 2026-11-17
- Tencent's DSpine speculative decoding ships GGUFs and a llama.cpp PR: 4.5x speedup on 27B — pmttyji · 2026-11-17
- Founder lands 1000 GPUs (40% GB300, 60% B300) on short-term contracts, dismissing the GPU shortage narrative — SumitGup · 2026-11-17