vLLM details MiniMax M3 optimization on AMD MI355X: 4.45x per-GPU throughput gains
vllm_project · x · 2026-09-11
The vLLM team, with AMD and EmbeddedLLM, published a deep dive on optimizing MiniMax M3 on Instinct MI355X after day-0 support, following the bottleneck as it moves.
Key results from the SemiAnalysis InferenceX benchmark:
- MXFP8 standard serving at concurrency 32 rose from 109.1 to 342.4 output tokens/s/GPU (3.14x); median TTFT fell from 1.46s to 0.67s
- At concurrency 128, the TP4/EP1 path reached 623.7 (2.09x)
- MXFP4 with TP2/EP1 hit 943.5 tokens/s/GPU — 4.45x the initial per-GPU result
- EAGLE3 speculative decoding reached 682.4; P/D disaggregation delivered 6,370.5 total tokens/s/GPU at concurrency 512
The post covers MSA sparse attention, quantization, topology tuning, and reusable performance tips for future model optimizations.
Related event: vLLM and AMD boost MiniMax M3 throughput up to 4.45x on MI355X(2 posts)→
More from Infra
- SemiAnalysis: OpenAI reportedly negotiating ~1GW TPU commitment while seeking sites — dylan522p · 2026-10-28
- I tested 11 video upscaling methods for AI video: local, Topaz and cloud compared — lopas99 · 2026-10-28
- Thermo-backed CTO says AI is in its dial-up phase, touts disaggregated compute for an ultrafast 'broadband moment' — beffjezos · 2026-10-28
- DeepSeek-V4.1-Flash Q2 runs at 41 tok/s on one RTX 3090, 90% token match vs API — musicurgy · 2026-10-28
- USearch v3 RC ships after 1,000 days, its libraries now embedded in 550K-star projects — viglovikov · 2026-10-28
- SK Hynix says memory demand will outpace supply through 2030 — Chemical_Hawk_6307 · 2026-10-28