SGLang ships day-0 Kimi K3.1 Pro support: 31K+ tok/s per GPU on GB300 NVL72
ying11231 · x · 2026-11-06
SGLang, together with radixark, NVIDIA, AMD, Moonshot and SemiAnalysis, announced day-0 inference support for Kimi K3.1 Pro, with docs and Cookbook recipes already available.
Key details:
- Kimi K3.1 Pro has 2.8T parameters (104B active) with 1M context.
- On GB300 NVL72: 420+ tok/s per user at low concurrency and 31K+ tok/s per GPU at high concurrency.
- Day-0 performance curves for B300, GB200 NVL72, B200 and AMD MI355X.
- SemiAnalysis highlighted out-of-the-box megakernel support.
- Supported hardware spans Vera Rubin, Blackwell, Hopper, AMD Instinct, PPU and Google TPU; deployment uses the kimi-k3.1-day-0 branch and dedicated Docker images.
More from Infra
- Muon optimizer is cheap in pretraining but adds up to 54% FLOPs in fine-tuning — kastnerkyle · 2026-11-06
- Micron's $134bn RPOs suggest the 17% selloff mispriced an AI demand break — tengyanAI · 2026-11-06
- Vivek Wadhwa: Big Tech's trillion-dollar bet on data centers is a mistake — sanjaykalra · 2026-11-06
- Is AMD 7900 XT/XTX viable for video generation in late 2026? — Beneficial_Fish_7509 · 2026-11-06
- NVKE open-source project wraps NVIDIA GPU clusters into a Kubernetes engine — gkapur · 2026-11-06
- Blue Origin unveils Northstar engine: 715,000 lbf thrust, 30% lighter than BE-4 — beffjezos · 2026-11-06