Baseten hits ~2,500 tok/s on DeepSeek v4.1 Flash with Vera Rubin, 1.7x faster than GB300
baseten · x · 2026-11-17
Baseten shared new low-latency results for DeepSeek v4.1 Flash: 2,500 tok/s on Vera Rubin at batch size 1, TP4, and 1.9ms per step — up to 1.7x faster than its previous GB300 numbers. The team is working with SGLang and radixark to upstream the optimizations into open source, with results presented by Tri Dao and others at last week's SGLang summit.
Related event: Baseten runs DeepSeek v4.1 Flash at ~2500 tok/s on Vera Rubin(2 posts)→
More from Infra
- Microsoft outage hits airlines and banks, sparking global cyber incident fears — mallow610 · 2026-11-17
- Texas data center freeze faces first test: Core Scientific's $1.3B AMD build — edgarpavlovsky · 2026-11-17
- vLLM claims 90x throughput on Rubin as analysts ignore software-driven gains — bookwormengr · 2026-11-17
- Runway CTO Kamil Sindi to break down training and real-time serving of video foundation models at AI Dev NYC — DeepLearningAI · 2026-11-17
- Microsoft's 30-hour Outlook outage reignites debate over lost senior engineers — 新智元 · 2026-11-17
- Michael Burry's Oracle short pays off: stock down 17.4% since August — SumitGup · 2026-11-17