Baseten Open-Sources Rust+CUDA DeepSeek Inference Engine Built From Scratch
Baseten open-sourced a DeepSeek V4.1-Flash inference engine built from scratch in pure Rust and CUDA, without PyTorch or any DSL, achieving about 1740 TPS decoding on 4x B300 and 2060 TPS on 8x B300.
2026-11-14 ~ 2026-11-14 · 2 related posts
- Episode 1: Baseten Open-Sources Rust+CUDA DeepSeek Inference Engine Built From Scratch(2026-11-14, 2 posts)
- Episode 2: Baseten runs DeepSeek v4.1 Flash at ~2500 tok/s on Vera Rubin(2026-11-17, 2 posts)
- From-scratch Rust+CUDA inference engine for DeepSeek hits 1740 TPS on 4x B300 — saranormous · 2026-11-14
- Baseten open-sources a Rust/CUDA inference engine for DeepSeek V4.1-Flash: 2,060 TPS on 8xB300 — A_K_Nain · 2026-11-14