A Reddit explainer breaks down MoE, KV cache, MLA and KDA behind Kimi K3
MohamedKadri_ · reddit · 2026-07-27
A Reddit user published a two-part plain-English explainer of MoE and the KV-cache stack behind Kimi K3, with references to the underlying papers and a where-used map for each claim.
The post walks through:
- the MoE sparsity race, from Mixtral and DeepSeek-V3 to Llama 4 Maverick, gpt-oss-120b, and K3’s company-reported 16-of-896-expert claim;
- why experts are not neat subject specialists, but routing patterns for code, syntax and similar features;
- why KV cache is the real long-context cost, including a worked example showing standard MHA would need about 131 GB for a 32k-token chat on V3’s published dimensions;
- how MLA cuts cache use by 93.3% versus the prior model, and how KDA uses a fixed-size state, per-channel forgetting, and a 3:1 hybrid to cut it by up to 75%.
The author says the explanation borrows from community analogies such as the “giant supermarket, small basket” framing and invites corrections.
More from Models
- Users Report Severe Downgrade in Opus 5: Hallucinations and Math Errors — whatsallthiss · 2026-07-27
- Reddit asks which local model works best for coding, planning and VS Code workflows — naunen · 2026-07-27
- GPT-5.6 Sol fixes 31 of 105 hidden bugs in a two-repo benchmark — PawelHuryn · 2026-07-27
- Opus 5's ARC-AGI-3 Leap Debunked as 'Complete Slop' Due to Benchmark Design Flaws — scaling01 · 2026-07-27
- Inkling still flirts back on simpler prompts, WeirdChat screenshot shows — ChowdhuryNeil · 2026-07-27
- Leaked Kimi K3.1 claims GPT-5.6-level performance with faster inference and open weights — iamaliveix · 2026-07-27