Baseten open-sources Rust/CUDA DeepSeek engine; agent on Vera Rubin finds optimizations speeding Blackwell by 10%

baseten · x · 2026-11-17

Baseten built an open-source inference engine for DeepSeek V4.1-Flash from scratch in Rust and CUDA: 1,740 TPS decode on 4×B300, 2,060 TPS on 8×B300, and 1M-token prefill in 12s. An engineer then repeated the exercise on the sparsely documented next-gen Vera Rubin architecture: the agent beat initial performance predictions and even found optimizations that generalized back to Blackwell, making B300 step time 10% faster. The author argues that since Opus 5.5, performance engineering has hit an inflection point — with solid verification, agents can hillclimb indefinitely while humans just encourage and steer.

Original post →

More from coding & agent

coding & agent channel →