FULL STORY

Baseten Builds Open-Source DeepSeek Engine, Pushes New Hardware

Baseten open-sourced a from-scratch Rust+CUDA inference engine for DeepSeek V4.1-Flash, then deployed the model on Nvidia's Vera Rubin hardware, reaching roughly 2500 tok/s.

2026-11-14 ~ 2026-11-17 · 2 episodes · 4 posts

Episode 1 · Baseten Open-Sources Rust+CUDA DeepSeek Inference Engine Built From Scratch (2026-11-14, 2 posts)

Baseten open-sourced a DeepSeek V4.1-Flash inference engine built from scratch in pure Rust and CUDA, without PyTorch or any DSL, achieving about 1740 TPS decoding on 4x B300 and 2060 TPS on 8x B300.

Episode 2 · Baseten runs DeepSeek v4.1 Flash at ~2500 tok/s on Vera Rubin (2026-11-17, 2 posts)

Baseten reported that DeepSeek v4.1 Flash achieves about 2500 tokens per second on Nvidia's Vera Rubin hardware using batch size 1 and TP4 configuration, showcasing its latest low-latency inference optimizations.