NVIDIA’s Dynamo docs add Kimi-K3 deployment recipes for GB200 and GB300
woosuk_k · x · 2026-07-29
NVIDIA’s Dynamo docs now include a deployment guide for Moonshot AI’s Kimi-K3.
The page describes serving Kimi-K3 on GB200 or GB300 with Dynamo + vLLM, using either aggregated or prefill/decode-disaggregated topologies. It highlights a multimodal MoE model with up to 1M-token context, KV-aware routing, MXFP4-packed routed experts, FP8 KV cache, and NVLink-based transfer and tensor parallelism on NVL72 racks.
More from Infra
- Meta's commitments jumped to $349B in a quarter as server memory prices tripled ahead of Oct 21 earnings — tengyanAI · 2026-10-16
- Meta releases Muse Spark Open, trained with vLLM-powered RL inference from day one — colinmcnamara · 2026-10-16
- Meta's open Muse Spark needs 493GB of memory — far beyond any consumer laptop — TejasKumar_ · 2026-10-16
- CoreWeave plunges 20% as credit markets price >60% default odds; is this dot-com all over again? — kimmonismus · 2026-10-16
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16