MiniMax M3.1 Flash tops 1K tok/s decode on SGLang, ~100K tok/s per GPU
ying11231 · x · 2026-10-24
SGLang, MiniMax, NVIDIA, AMD, and RadixArk ship Day-0 production support for MiniMax M3.1 Flash: >1K tok/s bs1 decode, 100K total tok/s per GPU on the AgentX coding-agent trace (96 concurrent sessions), broad support for NVIDIA Blackwell/Hopper and AMD MI350X/MI355X, speculative decoding via DSpark and DFlash2 drafts, HiCache, multimodal input, and 2.15× faster single-request decode with MegaKernels.
Related event: MiniMax M3.1 Flash Launches with Day 0 SGLang Support, Exceeding 1000 tok/s(2 posts)→
More from Infra
- Google's TPU outsourcing could hit $100B-$250B by 2028, but HBM is the bottleneck — Beth_Kindig · 2026-10-24
- Spotify details LLM podcast recommendations: SGLang multi-item scoring cuts per-candidate request fan-out — ying11231 · 2026-10-24
- NVIDIA's Nemotron 3.5 Bolt lands on Ollama: small VLM with 128K context — ollama · 2026-10-24
- Bittensor Ecosystem Weekly: $30M in GPU Hosting at 91% Utilization, 10x Cheaper Video Model — markjeffrey · 2026-10-24
- Stratechery: Apple×Meta Edge AI Partnership, Netflix's 40% Stock Slump, Stripe Sponsorship — Stratechery · 2026-10-24
- llama.cpp team and Hugging Face launch Llama: a 1MB desktop app to run open models locally — mervenoyann · 2026-10-24