Kimi K3 Architecture and 2.8T Scale
scaling01 · x · 2026-07-16
This share notes that Kimi K3 incorporates two architectural designs: Kimi Delta Attention (KDA) and Attention Residuals (AttnRes).
Additionally, cited sources confirm that Moonshot AI has verified the Kimi scale reaches 2.8T. The focus here is strictly on model architecture and scale, rather than application layers.
Related event: Kimi K3 Debuts Strong, Narrowing the Open-Weight Gap(184 posts)→
More from Models
- Developer Building a Unified Leaderboard of All Model Benchmark Scores — airesearch12 · 2026-09-11
- Rumor claims Kimi faked performance by serving Claude; DeepSeek new model surprises in evals — realsohamparekh · 2026-09-11
- GPT-5.6 writes well but is instantly forgettable, user complains — BasedRaddka · 2026-09-11
- Opus Refuses Protein Research Codebase Over 'Safety' Concerns, Dev Considers Rolling His Own — josephdviviano · 2026-09-11
- User Hails Unconfirmed 'DeepSeek 4.1 Flash' as an Inflection Point in LLMs — himanshustwts · 2026-09-11
- Terminal Bench v4: GLM-5.3 Leads at 41.9%, Kimi-K3 Underwhelms at 12.6% — Ok_Warning2146 · 2026-09-11