Kimi K3 uses QAT, RL and MOPD across a wide expert-task mix

nrehiew_ · x · 2026-07-29

The thread outlines Kimi K3’s training pipeline and task mix.

The pipeline is described as SFT with QAT using MXFP4 weights and MXFP8 activations, followed by RL and then MOPD.
They train many expert models, including one for each reasoning level.
The poster notes that the RL algorithm appears to be a GRPO variant rather than GLM 5.2 PPO.
MOPD is used to consolidate domain-specialized capabilities, but top-k distillation did not show a clear advantage.
The task categories include general verifiable tasks, kernel optimization, personal assistant tasks, autoresearch, and web development tasks, with SVG/3D explicitly mentioned.

More from Models