Moonshot’s Kimi K3 long-context design draws skepticism over 1M-token scaling
teortaxesTex · x · 2026-07-21
A thread questions whether Moonshot’s long-context research path is the right one, arguing that its post-R1 shift to MLA may be weak beyond 262K tokens and asking whether KDA can scale to 1M as well as CSA+HCA.
The attached architecture image shows Kimi K3’s design stack, including Stable LatentMoE, Gated MLA, KDA, and the Kimi Delta Attention backbone. The discussion frames long context as the AI era’s version of RAM and suggests the real moat may be organizational as much as model-level.
More from Models
- Fable 5.5 launch rumors persist for third week, but no frontier model drops this month — patience_cave · 2026-10-26
- Abliterated Model Flops in the Wild: It Got Worried About My Mental Health Instead — ryunuck · 2026-10-26
- OpenAI upgrades Decisions API: 400 questions per call, native voice and image support — OpenAIDevs · 2026-10-26
- Fangcun AI, China's first independent frontier AI safety lab, launches with DeepSeek eval report — JacquesThibs · 2026-10-26
- When AI hits an existential crisis: model confuses itself and everyone else — Ok-Delivery-7437 · 2026-10-26
- Kimi K3.1 Pro tops MathArena open models, closing frontier gap to about 6 months — teortaxesTex · 2026-10-26