vLLM ships Day-0 support for Kimi K3.1 Pro with sparse attention and FP8 KV cache
vLLM Blog · rss · 2026-11-04
vLLM announced Day-0 support for Kimi K3.1 Pro, with several serving optimizations: sparse attention on the model's KDA hybrid architecture, a packed KV cache using FP8 or NVFP4 latents with MXFP4 index keys, token-sharded prefill, decode context parallelism, and fused low-latency decode—enabling efficient deployment on release day.
More from Infra
- citrini: being part of AI supply chain no longer guarantees 100% yearly stock gains — citrini · 2026-11-05
- Building orbital data centers to dodge regulators is 'completely unserious' — zacharynado · 2026-11-05
- Verda Launches Inference Preview with Day-0 Access to Kimi K3.1 Pro — Kimi_Moonshot · 2026-11-05
- Banach: open-source engine streams 1T-parameter Kimi K2 from NVMe on a 64GB desktop — gautamrbharadwaj · 2026-11-05
- Google unveils $10B, 200-hectare datacenter project in Châteauroux, France with up to 1.2GW — QuintinPope5 · 2026-11-05
- Corebrum OS adds built-in VNC screen share, demoed remote-viewing an NVIDIA Thor robot from a Mac — chrismatthieu · 2026-11-05