DeepSeek v4.1 Flash hits ~2,500 tok/s on Vera Rubin in Baseten low-latency tests
baseten · x · 2026-11-17
Baseten shares new low-latency results for DeepSeek v4.1 Flash on Nvidia's Vera Rubin: 2,500 tok/s at batch size 1 with TP4 and 1.9ms per step — about 1.5x faster than their GB300 numbers. They are working with SGLang (sglproject / radixark) to bring the optimizations to open source, with Tri Dao and stel4th presenting the work at the SGLang summit.
More from Infra
- ARK Newsletter Flags Jensen Huang's Resignation as Major Test for Nvidia — downingARK · 2026-11-17
- Tracing Nvidia's circular financing: $170B in sales, only ~$35B unsettled — tengyanAI · 2026-11-17
- One Chase software bug rippled across US banking: a case for resilient infrastructure — AryHHAry · 2026-11-17
- Together Deleted Its Megakernels: AI Agents Tune Inference 28% Faster on B200 — togethercompute · 2026-11-17
- United Airlines hit by 'widespread system outage' days after national ground stop — shaunralston · 2026-11-17
- Cloudflare tops 10 million developers on its platform, more than doubling in a year — Beth_Kindig · 2026-11-17