Baseten hits ~2,500 tok/s on DeepSeek v4.1 Flash with Vera Rubin, 1.7x faster than GB300

baseten · x · 2026-11-17

Baseten shared new low-latency results for DeepSeek v4.1 Flash: 2,500 tok/s on Vera Rubin at batch size 1, TP4, and 1.9ms per step — up to 1.7x faster than its previous GB300 numbers. The team is working with SGLang and radixark to upstream the optimizations into open source, with results presented by Tri Dao and others at last week's SGLang summit.

Related event: Baseten runs DeepSeek v4.1 Flash at ~2500 tok/s on Vera Rubin(2 posts)→

Original post →

More from Infra

Infra channel →