Estimating Kimi K3 Active Parameters

teortaxesTex · x · 2026-07-17

This post questions how a model's active parameter count fluctuates, referencing a rough estimate:

• The model is estimated to have around 55–70B active parameters.
• This back-of-envelope calculation is based on "61 layers, 2K expert hidden dim, 2.8T parameters / 896 experts."
• The math roughly breaks down to: 50 million parameters per expert per layer. If 16 experts are activated per token, adding attention, shared layers, and embedding, the overall active compute sits around 60B.
• The original poster is asking how this number might change subsequently and whether it will land around 15–20GB or another range.

Original post →

More from Infra

• Nearly half of Claude revenue now flows through cloud marketplace commitments, up from 11% — BenBajarin · 2026-10-26
• Anthropic ARR goes vertical after price cuts: tokens up 4x, $100B+ ARR back in play — GavinSBaker · 2026-10-26
• Alex Banks argues power caps GPU supply, real risk is demand cooling first — thealexbanks · 2026-10-26
• Analyst: Ant has the most fungible AI workload of any model provider today — BenBajarin · 2026-10-26
• Anthropic S-1: training infra edge cut per-task cost 40% and doubled revenue growth — BenBajarin · 2026-10-26
• Rewriting scaling law study: optimal synthetic data share rises from 10% to 32% with compute — heghbalz · 2026-10-26

Infra channel →