Flint 40B benchmarked: 206 tok/s per user on a single RTX PRO 6000 Blackwell
dee_hw · x · 2026-10-23
- Autonomous Labs tested the freshly released Flint 40B on a 2× RTX PRO 6000 Blackwell box (96GB each, 192GB total, vLLM 0.26.1).
- With NVFP4 + SGLang on one card: 1,240 tok/s total output across 16 concurrent users (77.5 tok/s per user), 206 tok/s single-user, 0.3s 8K prompt prefill, and a 1.07M-token KV pool.
- Locally measured, no live traffic; dual-card results not reported. Shows 40B-class models are now comfortably servable from workstation hardware.
More from Infra
- Ex-Googler with 35 Years in Software Pursues Local AI Coding and Music Video Generation — julianharris · 2026-10-23
- Intelligence too cheap to meter: Opus price down 4x while AA score jumps 21 to 58 — haider1 · 2026-10-23
- SK Hyper in talks to fund envisioned $350B AI data center buildout in South Korea — dinabass · 2026-10-23
- MiniMax M3.1 Flash hits >1K tok/s decode with day-0 SGLang support across NVIDIA and AMD — MiniMax_AI · 2026-10-23
- MiniMax-M3.1-Flash goes open weights with day-0 NVIDIA NeMo fine-tuning up to 128 GPUs — MiniMax_AI · 2026-10-23
- RadixArk caught paying KOLs to coordinate quote-posts of SGLang announcement — EugeneVinitsky · 2026-10-23