Baseten open-sources a Rust/CUDA inference engine for DeepSeek V4.1-Flash: 2,060 TPS on 8xB300
A_K_Nain · x · 2026-11-14
Baseten built an inference engine for DeepSeek V4.1-Flash from scratch in Rust and CUDA, and open-sourced it. Performance: 1,740 TPS decode on 4xB300, 2,060 TPS decode on 8xB300, and 1M-token prefill in 12 seconds. Commenters welcomed the competition plus open source.
More from Infra
- Predicts Upcoming Devkit Card Will Be Best for Local AI Inference in 2 Years — gnukeith · 2026-11-14
- Unsloth's 5-10% quant gain was measured at 256K context — re-check your budget — petrusenko_max · 2026-11-14
- Running MiniMax H3 on a 16GB RX 9070 XT with RAM offload, configs open-sourced — NigaTroubles · 2026-11-14
- United CEO calls Starlink Wi-Fi the airline's biggest edge in 100-year history — Aiden_Tech_Ai · 2026-11-14
- Cloudflare Open-Sources Kitesurf: A Chromium-Free Headless Browser Built for Agents — alexcovo_eth · 2026-11-14
- xAI halves data center build times, Memphis campus to grow from 0.4 GW to 2.4 GW — elonmusk · 2026-11-14