Baseten open-sources Rust/CUDA DeepSeek engine; agent on Vera Rubin finds optimizations speeding Blackwell by 10%
baseten · x · 2026-11-17
Baseten built an open-source inference engine for DeepSeek V4.1-Flash from scratch in Rust and CUDA: 1,740 TPS decode on 4×B300, 2,060 TPS on 8×B300, and 1M-token prefill in 12s. An engineer then repeated the exercise on the sparsely documented next-gen Vera Rubin architecture: the agent beat initial performance predictions and even found optimizations that generalized back to Blackwell, making B300 step time 10% faster. The author argues that since Opus 5.5, performance engineering has hit an inflection point — with solid verification, agents can hillclimb indefinitely while humans just encourage and steer.
More from coding & agent
- Decision Models upgrades playground with image input, showing probabilities and per-request cost — airesearch12 · 2026-11-17
- Microsoft owns every repo on GitHub—so why does Copilot still suck at coding? — prasenx · 2026-11-17
- shadcn's underrated tip: ask your coding agent "are you sure?" before it writes any code — shadcn · 2026-11-17
- Stanford's CS329A Self-Improving AI Agents course: 9 free lectures, 10+ hours on YouTube — techNmak · 2026-11-17
- 'Stuck at 99%': a single-file HTML toy progress bar with fully synthesized Web Audio — prasenx · 2026-11-17
- The constellation view: monitoring agents as stars — green, yellow, red, gray — kevinkern · 2026-11-17