Moonshot Releases 2.8T Open-Weights Model Kimi K3

Moonshot AI has released Kimi K3, an open-weights MoE model with 2.8 trillion total parameters. The model supports a 1 million token context and native multimodal inputs, with reasoning mode enabled by default. The release significantly narrows the gap between open-source and closed-source frontier models, marking a major milestone.

Architectural Innovations and Efficiency

According to technical details shared by @Ahmad_Al_Dahle, K3 achieved a 2.5x efficiency improvement over K2. Core architectural innovations include the KDA mechanism replacing Gated DeltaNet's single scalar decay with a learnable per-dimension forgetting mechanism, AttnRes enabling cross-depth selective retrieval, and a 16/896 MoE architecture. @zephyr_z9 added that K3's size equates to about 1.4T when served in fp4 precision, with a sparsity of only 1.7%, making it one of the sparsest frontier models.

Performance and Industry Impact

Based on Artificial Analysis data, @ImaginaryRea1ity noted that K3 shortened the gap between open and closed frontiers to about 1.5 months. @PeterDiamandis stated that K3 ranked first in multiple tasks like front-end programming, marketing, design, and data analysis. @Ahmad_Al_Dahle suggested that if open-source labs maintain this efficiency, brute-force compute advantages will depreciate rapidly. @mishig25 reposted researcher Nathan Lambert's view that Chinese labs now hold 3 of the top 8 smartest models. @emollick and @markjeffrey noted that China is almost the only player left in frontier open-weights, as the US and Europe lack the incentive to invest, and even former OpenAI CTO Mira Murati's new company (Inkling) is far from this level.

Commercial Potential and Deployment Barriers

Commercially, @zephyr_z9 estimated a gross margin of at least 75%–80% for its inference business. However, K3's massive size imposes extremely high hardware barriers. @teortaxesTex mentioned that deploying the model would likely require a full rack of 64 high-end chips. Furthermore, @davidyin44 relayed Nathan Lambert's perspective that K3's open-source strategy has essentially escalated the open-weights war.

2026-07-19 ~ 2026-07-21 · 14 related posts

Full story(20 episodes)→