Kimi K3’s QB routing uses quantiles to balance MoE expert load
stochasticchasm · x · 2026-07-28
A post about Kimi K3’s Quantile Balancing (QB) routing update.
- The author says QB scale-up is welcome and notes it follows DeepSeek-style normalization, rather than using biased scores directly for weighting.
- They like QB’s causal setup: it estimates current thresholds from the previous batch.
- The screenshot explains the method in the Kimi K3 technical report:
- replace standard Top-k routing with Top-(k+1) on biased router scores
- use the extra position as a cutoff threshold
- set each expert’s bias by the relevant quantile of token-score margins
- The goal is better load balancing for large MoE routing, especially as expert pools grow.
Related event: Kimi K3 Technical Report Details MoE Architecture and SiTU-GLU Design(4 posts)→
More from Models
- Moonshot releases weights for a 2.8T frontier model and more run-time infrastructure — 0xSammy · 2026-07-28
- Kimi K3 report shows 2.5× better scaling efficiency than Kimi K2 — ricklamers · 2026-07-28
- The Verge says Moonshot’s open Kimi K3 could undercut closed U.S. AI models — The Verge AI · 2026-07-28
- Moonshot releases Kimi K3 with 2.8T MoE, 1M context and 2.5× efficiency gain — joeddav · 2026-07-28
- Kimi K3 lands on Fireworks AI for inference and training — omarsar0 · 2026-07-28
- Gemini video generation adds words and blocks some harmless prompts — Individual-Cookie615 · 2026-07-28