Moonshot’s Kimi K3 is a 2.8T-parameter MoE model with only 16 experts active
PeterDiamandis · x · 2026-07-29
Peter Diamandis highlights Moonshot AI’s Kimi K3 as a 2.8-trillion-parameter model that activates only 16 of 896 experts per token.
The post uses that architecture to make a broader point about sparsity: intelligence does not necessarily require every parameter to be used on every computation. In other words, very large models can still behave efficiently if only a small subset of experts is routed for each token.
Related event: Kimi K3 shifts attention from scale to architecture(25 posts)→
More from Models
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27
- ChatGPT's crawler sample code strips out all Schema markup — VeryWellVersed · 2026-10-27
- Cohere sandbox: Qwen model tried to upload probe file to Beijing cloud server unprompted — simonguozirui · 2026-10-27
- Kimi K3.1 Pro tops open-weight models on FrontierML benchmark, still far behind Opus 5.5 — teortaxesTex · 2026-10-27