vLLM ships day-0 serving support for Moonshot’s 2.8T-parameter Kimi K3
vllm_project · x · 2026-07-28
vLLM publishes a day-0 serving guide for Moonshot’s Kimi K3
vLLM says it now supports Kimi K3 on day 0 as Moonshot’s weights are public. The post is a practical deployment guide for running the model in production.
Key details from the guide:
- Kimi K3 is a 2.8T-parameter MoE model with 16 of 896 experts active per token.
- It uses Kimi Delta Attention (KDA) and Attention Residuals (AttnRes).
- The model has a 1M-token context window and native vision.
- The engineering challenge is making KDA, MXFP4 MoE execution, KV cache management, prefill/decode disaggregation, speculative decoding, and long-context serving work together.
The attached diagrams focus on cache layout, decode throughput, and the memory/performance tradeoffs of the model’s architecture.
Related event: vLLM Provides Day-0 Support for 2.8T Parameter Kimi K3(2 posts)→
More from Infra
- Compute, not algorithms, is the real moat in frontier AI — GavinSBaker · 2026-07-28
- Renting GPUs and open-weight models cut one AI bill from $1.2M to $100K — kimmonismus · 2026-07-28
- Running Kimi-K3 Locally Costs Up to 1 Million Euros, Jokes German Tech Blogger — FlorianGallwitz · 2026-07-28
- Bittensor updates $TAO emissions with a 61% gate that favors top-valued subnets — markjeffrey · 2026-07-28
- Qualcomm says NPU latency beats GPU and CPU for on-device AI agents — qdrant_engine · 2026-07-28
- MLA-based KV cache costs 12 GB per million tokens, with KDA state at 230 MB BF16 — zephyr_z9 · 2026-07-28