Leak: Moonshot to Release Kimi K3 with 2.8T Parameters and 1M Context
johnseach · x · 2026-07-27
According to leaks, Moonshot AI is set to open-source the Kimi K3 model on Hugging Face today, marking the world's first 3-trillion parameter open-weight model.
Core Specs & Architecture:
- Total parameters reach 2.8 trillion, utilizing a highly sparse MoE architecture (only 16 of 896 experts active per token).
- Supports a 1 million token context window with native vision support.
- Built on Kimi Delta Attention (KDA) + Attention Residuals, designed for long-horizon coding, agentic workflows, and deep knowledge work.
Hardware & Deployment:
- Native MXFP4 weights are 594 GB, requiring a minimum of 8×H100 80GB for experimental loading, and 64+ accelerators for serious production serving.
- Expected to use a Modified MIT license (commercial use allowed), with vLLM providing day-0 support.
This marks another major breakthrough for open-source models, with Chinese labs continuing to lead in shipping massive open weights.
More from Infra
- Min-p 0.05 restores quality in Laguna S2.1 and should be the default — antirez · 2026-07-27
- Cloudflare says its verification screen can now identify AI agents too — shashib · 2026-07-27
- Qwen3.6-27B speculative decoding speeds up as quantization gets heavier — thavoc77 · 2026-07-27
- AgentPond adds Supabase support and keeps agent traces inside the app — MarcusSchiesser · 2026-07-27
- A forked SGLang stack brings Qwen and Laguna to 4x V100 GPUs — Primary_Exchange21 · 2026-07-27
- ARM shows neural super sampling and denoising at SIGGRAPH — palgorithm · 2026-07-27