vLLM ships Day-0 support for MiniMax-M3.1-Flash with 10.2x throughput on Rubin NVL72
vllm_project · x · 2026-11-12
The vLLM team announced Day-0 support for MiniMax-M3.1-Flash across NVIDIA Blackwell, Rubin (SM107) and AMD MI350-series GPUs.
Model and inference features
- M3.1-Flash improves reasoning, coding, agentic and visual capabilities
- Native NVFP4 KV cache (indexer and attention path), NVFP4-quantized MoE weights, and a DSpark draft model for speculative decoding
Blackwell optimizations
- MSA sparse attention kernels use CUDA 13.4's QMUL4 instruction to dequantize NVFP4 to FP8 in-kernel: up to 1.7x faster indexer QK decode, 1.3x faster sparse attention decode
- NVFP4 MegaMoE with MXFP8 shared-expert fusion and early TMEM release: 20% lower prefill kernel latency, 7% higher prefill throughput
- Indexer decode context parallelism: up to 3.3x faster indexer per sparse layer at 512K context on 4x GB300
- QKV GEMM + norm + RoPE + KV-insert fusion: 1.1–1.6x faster prefill at 512+ tokens, 19% lower P50 TTFT at TP4
- FlashInfer PrimiTS backend for the DSpark drafter: 1.3–2x faster than FlashAttention 4
Rubin and AMD
- Rubin MSA kernels are integrated into vLLM; on SemiAnalysis AgentX, Rubin NVL72 vs GB300 NVL72 shows 10.2x total tok/s/chip at 546 P90 tok/s/user, 6.5x at 376 P90, and up to 1,205 P90 tok/s/user at low concurrency
- AMD MI350: NVFP4-native MSA kernel + DCP + dual-batch overlap + mono kernel deliver 2–3x gains across the board
Related event: vLLM Benchmarks Show Rubin NVL72 Delivering 10.2x GB300 Throughput(2 posts)→
More from Infra
- AMD surges 11% on unverified report Nvidia board prepares to oust CEO Jensen Huang — Polymarket · 2026-11-12
- celld self-hosts Workers, Durable Objects and queues using just an object store bucket — irvinebroque · 2026-11-12
- Azar: transparent chip diffusion strategy backfired as China pivots models to Huawei chips — beffjezos · 2026-11-12
- Running a minimal local agent on 24GB RAM: Pi Agent + Bonsai 2 9B hits ~30 tok/s — Potential_Block4598 · 2026-11-12
- DigitalOcean: Many LLM Serving "Wins" Are Noise — One +4% Winner Wasn't Real — AI Engineer · 2026-11-12
- Homemade inference box pairs AMD CPUs with Huawei Ascend 950PR NPUs — AIFlow_ML · 2026-11-12