Kimi K3’s constant-state design cuts long-context memory use by about 73%
bookwormengr · x · 2026-07-29
The post analyzes Kimi K3’s constant-state design and argues that its long-context memory savings may matter more than critics admit.
- The image compares memory use for 128K and 1M tokens: All-MLA (93L) uses 13.7 GB and 107 GB respectively, while Kimi K3 (24 MLA + KDA state) uses 3.5 GB + 0.22 GB and 29.0 GB + 0.22 GB.
- The core claim is that Kimi K3 avoids linearly growing KV cache for most of the conversation by using Kimi Delta Attention (KDA) / shared-key-value attention variants.
- The post frames this as a rebuttal to the “it’s already priced in” argument: if long-context workloads become much cheaper on memory, the pricing and demand dynamics for memory chips and related infrastructure could change materially.
- It also notes that the debate on X is not just about architecture novelty, but about whether long-context inference economics have been underestimated.
Related event: Kimi K3 shifts attention from scale to architecture(25 posts)→
More from Infra
- Meta's open Muse Spark needs 493GB of memory — far beyond any consumer laptop — TejasKumar_ · 2026-10-16
- CoreWeave plunges 20% as credit markets price >60% default odds; is this dot-com all over again? — kimmonismus · 2026-10-16
- MiniCPM-V 4.7 35B-A3B hits 17+ tokens/s on iPad with ~3GB memory — techNmak · 2026-10-16
- China rolls out 'token factories' and a 2026-28 token economy action plan — nordicinst · 2026-10-16
- Vercel Sandbox Drives go GA: attach TBs of data in sub-second, fork instantly — cramforce · 2026-10-16
- NVIDIA launches RTX Spark for local AI on Windows PCs — ostrisai · 2026-10-16