Baseten pushes DeepSeek V4.1 Flash to ~2,500 tok/s on Vera Rubin, 1.7x faster than GB300

ying11231 · x · 2026-11-17

Baseten shared new low-latency results for DeepSeek V4.1 Flash: 2,500 tok/s on Vera Rubin at batch size 1 with TP4, just 1.9ms per step — up to 1.7x faster than its previous GB300 numbers. The team is working with the SGLang project and radixark to upstream the optimizations into open source; details were presented by Tri Dao and colleagues at last week's SGLang summit.

Related event: Baseten Pushes DeepSeek V4.1 Flash to ~2500 tok/s on Vera Rubin(3 posts)→

Original post →

More from Infra

Infra channel →