vLLM details M3.1 Flash serving: NVFP4 cuts weights 44.6%, boosts capacity 2.7x
vllm_project · x · 2026-10-23
vLLM's technical deep-dive (part 4/5) on serving MiniMax M3.1 Flash:
- Native NVFP4 shrinks the checkpoint from 451.5 GB (MXFP8) to 250.1 GB (-44.6%) and cuts KV memory per token by 43.75% vs FP8
- At 256K context on 4× B300 with DEP4, M3.1 fits 76 concurrent sequences vs 28 for M3 (2.7x capacity)
- CUDA 13.4's QMUL4 converts NVFP4 KV to FP8 directly: up to 1.7x faster indexer QK decode, 1.3x faster sparse-attention decode
- NVFP4 MegaMoE fuses MXFP8 shared experts and releases TMEM early for 20% lower prefill kernel latency; Indexer DCP hits 3.3x per-layer speedup at 512K on 4× GB300; fused QKV/norm/RoPE/KV insertion lowers p50 TTFT 19% at TP4
Related event: vLLM Announces Day-0 Support for MiniMax M3.1 Flash with NVFP4 Gains(4 posts)→
More from Infra
- NVIDIA's $1.2M HGX Rubin NVL8 hits GTC Berlin, promising 10x tokens per megawatt over B200 — thursdai_pod · 2026-10-23
- Amazon scraps plans for four data centers on $427M former GWU Ashburn campus — GaryMarcus · 2026-10-23
- Developer platforms raided for AI inference keys: Railway, Fireworks, LiteLLM all hit this month — ccerrato147 · 2026-10-23
- Koi Paper: Self-Calibrating Planner Boosts Heterogeneous LLM Fleet Goodput 1.18x, Cuts $/token 20% — burny_tech · 2026-10-23
- Inco Open-Sources DFlash 2 Drafter: Lossless 3.5× Speedup for MiniMax M3.1 Flash — Lianhuiq · 2026-10-23
- Cloudflare CEO: agent traffic will be 1000x human traffic in 5 years, internet needs micropayments — 4KTV · 2026-10-23