Baseten Open-Sources Rust+CUDA DeepSeek Inference Engine Built From Scratch

Baseten open-sourced a DeepSeek V4.1-Flash inference engine built from scratch in pure Rust and CUDA, without PyTorch or any DSL, achieving about 1740 TPS decoding on 4x B300 and 2060 TPS on 8x B300.

2026-11-14 ~ 2026-11-14 · 2 related posts

Full story(2 episodes)→