vLLM ships Day-0 support for Kimi K3.1: 61.2% on long-horizon coding bench, RL-ready stack
vllm_project · x · 2026-11-05
The vLLM project announced Day-0 support for Moonshot AI's Kimi K3.1, spanning both serving and RL training.
Key points
- Architecture: K3.1 combines DSA sparse attention (up to 2,048 key positions per query) with recurrent Kimi Delta Attention (KDA); one packed cache holds MLA latents, indexer keys, and KDA state.
- Serving work: token-sharded prefill, exact top-k merging for decode context parallelism, and a fused low-latency decode path; state handoff between prefill/decode workers must preserve both state forms.
- Long-horizon results: K3.1 Pro scores 61.2% on the internal Kimi Code Bench 3.0 Long-horizon, up from K3's 55.95% (+5.25 points), covering multi-step execution, tool use, and end-to-end engineering; both runs use the Claude Code harness at maximum reasoning effort.
- Training side: @inferact handled training–inference consistency for KDA plus cache/state management for long-running agents; vLLM + Megatron makes the open stack ready for both serving and continued training.
Related event: vLLM Ships Day-0 Support for Kimi K3.1 Across Training and Serving(5 posts)→
More from Infra
- US AI scale faces two physical gates: delivered power and manufacturing — ingliguori · 2026-11-05
- Liquid AI ships LFM 3 models (1.7B/2B) running locally on iPhone Duo via MLX — helloiamleonie · 2026-11-05
- Yuma backs Bittensor subnet Fugal claiming GPT-5.5-level results at 1/6 the cost — markjeffrey · 2026-11-05
- Speculative decoding makes Qwen3.6/3.8 much faster, latest draft model adds ~10% — teortaxesTex · 2026-11-05
- webAI open-sources 8.8B TwIL-8B hitting 93% of frontier coding performance — dr_cintas · 2026-11-05
- Together Compiler optimizes new open models for production in hours, not weeks — togethercompute · 2026-11-05