vLLM + MiniMax: 3.3x faster 512K-context indexer serving, 19% lower TTFT on GB300
vllm_project · x · 2026-10-19
vLLM detailed its serving-stack optimizations for MiniMax M3.1-Flash, built with MiniMax and Inferact, jointly optimizing long-context serving and speculative decoding:
- Indexer DCP partitions context across GPUs and exchanges local top-16 candidates over NVLink: up to 3.3× faster indexer execution per sparse layer at 512K context on 4× GB300.
- Fused QKV projection, norm, RoPE and KV insertion remove an HBM round trip: 19% lower p50 TTFT on GB300 at TP4.
- DSpark speculative decoding uses FlashInfer PrimiTS for draft attention: 1.3–2× faster than FlashAttention 4, with 5% lower overall forward time.
Context from the thread: Blackwell MSA kernels use CUDA 13.4's QMUL4 instruction to convert NVFP4 KV directly to FP8, avoiding the previous FP16 round trip—1.7× faster indexer QK decode and 1.3× faster sparse-attention decode at kernel level. These are separate measurements, not additive. CUDA and ROCm Docker images are available.
More from Infra
- vLLM 0.32.0 adds structured-decision API reading first-token probabilities — AccBalanced · 2026-10-19
- GPU pricing is now set by cost of capital, not spot rates, argues Maya — AccBalanced · 2026-10-19
- Liquid Inference ships DeepSeek V4.1-Flash support, ComfyUI plugin over weekend — AccBalanced · 2026-10-19
- Analyst revises 2027 datacenter forecast up to +50GW, citing satellite imagery — AccBalanced · 2026-10-19
- Cisco and NVIDIA unveil USFP optical module plan: 4x bandwidth density — shashib · 2026-10-19
- Hyperscaler capex shifting from chip vendors to in-house silicon by 2027, investor argues — pdamodaran · 2026-10-19