vLLM ships Day-0 serving for MiniMax M3.1 Flash: NVFP4 cuts KV memory 43.75%, 2.7× concurrency
vllm_project · x · 2026-10-23
vLLM announced Day-0 serving support for MiniMax M3.1 Flash, with AMD acceleration, DSpark speculative decoding, and RL training support.
Key numbers: - Native NVFP4 shrinks the checkpoint from 451.5GB (M3-MXFP8) to 250.1GB (-44.6%) and cuts per-token KV memory 43.75% vs FP8; at 256K context on 4× B300, concurrent sequences rise from 28 (M3) to 76 (2.7×); - AMD: on MI350X at TP4, mono decode is up to 2.3× faster and low-concurrency throughput doubles; - DSpark: FlashInfer PrimiTS draft attention is 1.3–2× faster than FlashAttention 4; long-context indexer up to 3.3×, p50 TTFT 19% lower; - RL: Day-0 veRL + FSDP2 + LoRA, training adapters against the NVFP4 checkpoint.
CUDA 13.4 recommended; full recipe and commands published.
Related event: vLLM Announces Day-0 Support for MiniMax M3.1 Flash with NVFP4 Gains(4 posts)→
More from Infra
- NVIDIA's $1.2M HGX Rubin NVL8 hits GTC Berlin, promising 10x tokens per megawatt over B200 — thursdai_pod · 2026-10-23
- Amazon scraps plans for four data centers on $427M former GWU Ashburn campus — GaryMarcus · 2026-10-23
- Developer platforms raided for AI inference keys: Railway, Fireworks, LiteLLM all hit this month — ccerrato147 · 2026-10-23
- Koi Paper: Self-Calibrating Planner Boosts Heterogeneous LLM Fleet Goodput 1.18x, Cuts $/token 20% — burny_tech · 2026-10-23
- Inco Open-Sources DFlash 2 Drafter: Lossless 3.5× Speedup for MiniMax M3.1 Flash — Lianhuiq · 2026-10-23
- Cloudflare CEO: agent traffic will be 1000x human traffic in 5 years, internet needs micropayments — 4KTV · 2026-10-23