Inside Kimi K3: 2.8T Parameters and Active Forgetting

Kimi's K3 emerges as the largest open-source model with 2.8 trillion total parameters, activating only 104 billion per inference. It innovatively replaces RoPE with KDA and LatentMoE, utilizing an active forgetting mechanism to solve long-context bottlenecks.

2026-07-30 ~ 2026-07-31 · 2 related posts

• Kimi K3 Architecture: Dropping RoPE for KDA and LatentMoE — khademinori · 2026-07-30
• Kimi K3 Architecture Explained: Building a 2.8T Parameter Open Model via 'Active Forgetting' — AccBalanced · 2026-07-31