FULL STORY
Baseten Builds Open-Source DeepSeek Engine, Pushes New Hardware
Baseten open-sourced a from-scratch Rust+CUDA inference engine for DeepSeek V4.1-Flash, then deployed the model on Nvidia's Vera Rubin hardware, reaching roughly 2500 tok/s.
2026-11-14 ~ 2026-11-17 · 2 episodes · 4 posts
Episode 1 · Baseten Open-Sources Rust+CUDA DeepSeek Inference Engine Built From Scratch (2026-11-14, 2 posts)
Baseten open-sourced a DeepSeek V4.1-Flash inference engine built from scratch in pure Rust and CUDA, without PyTorch or any DSL, achieving about 1740 TPS decoding on 4x B300 and 2060 TPS on 8x B300.
- From-scratch Rust+CUDA inference engine for DeepSeek hits 1740 TPS on 4x B300 — saranormous · 2026-11-14
- Baseten open-sources a Rust/CUDA inference engine for DeepSeek V4.1-Flash: 2,060 TPS on 8xB300 — A_K_Nain · 2026-11-14
Episode 2 · Baseten runs DeepSeek v4.1 Flash at ~2500 tok/s on Vera Rubin (2026-11-17, 2 posts)
Baseten reported that DeepSeek v4.1 Flash achieves about 2500 tokens per second on Nvidia's Vera Rubin hardware using batch size 1 and TP4 configuration, showcasing its latest low-latency inference optimizations.
- DeepSeek v4.1 Flash hits ~2,500 tok/s on Vera Rubin in Baseten low-latency tests — baseten · 2026-11-17
- Baseten hits ~2,500 tok/s on DeepSeek v4.1 Flash with Vera Rubin, 1.7x faster than GB300 — baseten · 2026-11-17