Estimating Kimi K3 Active Parameters

teortaxesTex · x · 2026-07-17

This post questions how a model's active parameter count fluctuates, referencing a rough estimate:

• The model is estimated to have around 55–70B active parameters.
• This back-of-envelope calculation is based on "61 layers, 2K expert hidden dim, 2.8T parameters / 896 experts."
• The math roughly breaks down to: 50 million parameters per expert per layer. If 16 experts are activated per token, adding attention, shared layers, and embedding, the overall active compute sits around 60B.
• The original poster is asking how this number might change subsequently and whether it will land around 15–20GB or another range.

Original post →

More from Infra

• Nothing phone mockup turns a film joke into a modular design meme — ZeYanjie · 2026-07-22
• Actual Computer says its inference stack is tuned for Nvidia’s consumer Blackwell lineup — markjeffrey · 2026-07-22
• Ben Bajarin says CPU demand is still being badly underestimated — BenBajarin · 2026-07-22
• An energy model says the U.S. could run short of natural gas starting in 2028 — churchkey · 2026-07-22
• Arbitrum fee simulation shows higher gas capacity but lower L2 revenue under ArbOS61 — tomwanhh · 2026-07-22
• NVIDIA pushes OpenUSD as the common layer for simulation and physical AI — MonaJalal_ · 2026-07-22

Infra channel →