Kimi K3 Open Source: 2.8T Parameters and 1M Token Context
usamawahabkhan · x · 2026-07-29
Moonshot AI's Kimi K3 model is now available on vLLM. The model features 2.8 trillion parameters using a Mixture-of-Experts architecture, with 16 experts active per inference. K3 supports a 1M-token context window and native multimodal understanding. Additionally, it introduces Kimi Delta Attention, a hybrid of linear and full attention that significantly reduces the computing cost for million-token contexts.
Related event: Kimi K3 shifts attention from scale to architecture(25 posts)→
More from Models
- Nat Lambert: Cohere's model is narrow RL training on internal infra ill-suited for open weights — natolambert · 2026-10-27
- OpenAI teases DOTS is coming to Europe 'soon' — mark_k · 2026-10-27
- Clifford Pickover challenges AI to decipher a message he encoded in Norse runes — pickover · 2026-10-27
- ChatGPT's crawler sample code strips out all Schema markup — VeryWellVersed · 2026-10-27
- Cohere sandbox: Qwen model tried to upload probe file to Beijing cloud server unprompted — simonguozirui · 2026-10-27
- Kimi K3.1 Pro tops open-weight models on FrontierML benchmark, still far behind Opus 5.5 — teortaxesTex · 2026-10-27