Kimi K3 Released: 2.8T Open Weights

Latent Space · rss · 2026-07-17

This AI News issue heavily focuses on Moonshot's release of Kimi K3:

Key Takeaways

Kimi K3 is positioned as a frontier-class open-weights model, marking a massive release in the open-source community.
Official specs include 2.8T total parameters, a 1M context window, native multimodal input, text output, with weights promised to be open-sourced before July 27.
Pricing is noted as close to Sonnet: $3 / 1M input tokens and $15 / 1M output tokens, with substantial discounts for cached inputs.

Technical Highlights

K3 utilizes Kimi Delta Attention (KDA), which officially claims up to 6.3x decoding acceleration for million-token scenarios.
It introduces Attention Residuals (AttnRes), reportedly boosting training efficiency by roughly 25% with under 2% additional cost.
The community discovered further details in the tech blog: potential use of LatentMoE / Stable LatentMoE with sparsely activated experts, along with mentions of Muon, load balancing, and a new activation function, SiTU.

Benchmarks & Rankings

Arena's early results after integrating K3 are highly impressive:
Ranked No. 1 in Frontend Code Arena with a reported score of 1679, surpassing Claude Fable 5.
Reached No. 9 in Text Arena, making the top ten in sub-categories like creative writing, coding, and instruction following.
Artificial Analysis's independent evaluation placed it at an AA Intelligence Index of 57, overall close to Opus 4.8 and GPT-5.5, though still behind Fable 5 and GPT-5.6 Sol.
They also provided task cost and token consumption data: the average task cost for the full Intelligence Index run was about $0.94, outputting 21% fewer tokens than K2.6.

Inference & Engineering

Early online API services observed speeds of around 26-28 tok/s, leading to speculation that speculative decoding might not be fully enabled yet.
Moonshot's blog suggests deploying on 64+ accelerator supernode configurations for optimal inference efficiency.
vLLM confirmed that Moonshot contributed the KDA prefix caching implementation upstream, indicating that this release also drove adaptations in serving and runtimes, beyond just the model itself.

Controversies & Caveats

The author of ProgramBench cautioned that certain statistical metrics might overestimate practical usability because non-recommended metrics were used.
The text explicitly notes that despite K3's strength, a gap remains in user experience compared to Claude Fable 5 and GPT-5.6 Sol.

More from coding & agent