DeepSeek v4.1 Flash hits ~2,500 tok/s on Vera Rubin in Baseten low-latency tests

baseten · x · 2026-11-17

Baseten shares new low-latency results for DeepSeek v4.1 Flash on Nvidia's Vera Rubin: 2,500 tok/s at batch size 1 with TP4 and 1.9ms per step — about 1.5x faster than their GB300 numbers. They are working with SGLang (sglproject / radixark) to bring the optimizations to open source, with Tri Dao and stel4th presenting the work at the SGLang summit.

Original post →

More from Infra

Infra channel →