vLLM Officially Supports Kimi K3 Deployment, Requires 8x GB300 Minimum
vllm_project · x · 2026-08-07
vLLM announces it is officially verified by Moonshot to deliver performant inference for the Kimi K3 model.
Model & Deployment Details:
- Architecture: Kimi K3 is a 2.8T-parameter native multimodal MoE model with 16 active experts per token (out of 896). It utilizes Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), supporting a 1M-token context window and native vision.
- Hardware: Requires at least 8x GB300 (multi-node for production) or 8x MI355X/MI350X (ROCm).
- Environment: Ships with a dedicated Docker image requiring CUDA 13 (cu130) and r580+ NVIDIA drivers.
- Optimizations: Supports TP8, TEP16, and disaggregated P/D profiles. Recommends deepepv2 backend for cross-node communication.
More from Infra
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16
- llama.cpp PR boosts CPU inference speed with merged AVX2 tiled accumulators — jacek2023 · 2026-10-16
- Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16
- Meme: driving to the chip factory in 2028 captures AI's compute anxiety — beffjezos · 2026-10-16
- One volunteer maintains the time zone file behind 4 billion phones — danbri · 2026-10-16