Kimi K3 uses fixed-size KDA state instead of a growing KV cache
vllm_project · x · 2026-07-27
- The post explains why fitting 2.8T parameters was not the hardest part for Kimi K3; caching them was.
- Many layers use a fixed-size KDA state instead of a growing KV cache, so there is no per-token KV to hash.
- Prefix caching was rebuilt around that design, and the author says every hybrid model after K3 can inherit the result.
- The attached diagram shows the model’s internal stack, including Kimi Delta Attention, Gated MLA, Stable LatentMoE, and the repeated block structure around KDA.
Related event: vLLM Analyzes Kimi K3: Caching 2.8T Parameters is the Real Challenge(2 posts)→
More from Infra
- Claude chat indexing incident sparks a push for confidential-compute AI — bittingthembits · 2026-07-27
- Moonshot open-sources MoonEP as open models vs closed labs debate intensifies — KyeGomezB · 2026-07-27
- Kimi K3’s 2.5x scaling-law gain draws praise for training efficiency — andrew_n_carr · 2026-07-27
- Kimi K3 goes live on Nebius with 1M-token context and a 57 AA score — teortaxesTex · 2026-07-27
- AI agent finds a longstanding Bun Node-compat bug in `child_process.spawn` — steipete · 2026-07-27
- llama.cpp adds support for Nanbeige4.2 in pull request 25994 — pmttyji · 2026-07-27