Kimi k3 RL details: length-weighted baselines and a cost-intelligence Pareto reward
kastnerkyle · x · 2026-09-13
An analysis of Kimi k3's RL training reveals two key choices: (1) they replaced GRPO's group-mean baseline with a length-weighted baseline — each rollout weighted by its length Lᵢ — for more stable rollout-trainer numerics; (2) the reward targets only cost (dollars and rollout time) and average solve rate, R = S - λe·C, where λe is chosen as the slope of the cost-intelligence Pareto curve at effort level e. Setting λe higher or lower just moves the model along the curve instead of advancing it.
More from Research
- AIR: Effective-Learning-Rate Scheduler Beats Cosine on 128B-Token MoE Pretraining — stochasticchasm · 2026-10-30
- NVIDIA researcher: train robot policies with privileged sim info, improving Pi 0.5 — wellecks · 2026-10-30
- CMU's Gym-Anything Auto-Builds 200+ Agent Environments from Any Software, Selected by GDP — wellecks · 2026-10-30
- CMU Auto-Builds 25 Robot Simulations with Agents; Frontier Model Solves 0 of 25 Test Tasks — wellecks · 2026-10-30
- Leveraging Simulator Privileged Information: Three Methods to Improve Robot Models — wellecks · 2026-10-30
- DeepMind's f_charton shares his take on the current state of AI and mathematics in Sorbonne talk — f_charton · 2026-10-30