Xiaomi MiMo RL Training Details Spark Debate: Suspected 1M-Context RL, Big Gains in Just 30 Steps

Xiaomi MiMo-V2.6's RL training details triggered intense discussion in overseas technical communities. stochasticchasm suspects this is the industry's first RL training on a 1M-token context, with dramatic gains in only 30 steps and no plateau on the training curve; tokenbender and nrehiew offered systematic breakdowns of the algorithmic modifications, training efficiency, and infra practices. The converging takeaway: large-batch, stabilized standard GRPO may deliver far higher sample efficiency than previously expected.

Confirmed

• MiMo uses a classic architecture, described by stochasticchasm as "simple and strong".
• Extremely large per-step batches: 1568 tasks × 16 rollouts per task, consuming enormous token counts per run.
• stochasticchasm noted the model improved dramatically in only 30 RL steps, far fewer than expected, with no plateau; he and commenter nrehiew questioned why training was not continued.
• Training uses a very standard GRPO form without a critic model. stochasticchasm guesses the large batch already suffices to reduce gradient noise, even without larger group sizes.
• tokenbender catalogued targeted GRPO modifications: prompt-mean loss over token-mean loss (DAPO) to avoid overweighting long answers; separate clipping bounds for positive/negative advantages. The philosophy: keep the algorithm simple, modify for stability and scale.
• tokenbender argued that high throughput, heterogeneous environments, and only 13% policy drift staleness imply most environments run or improve directly on base weights, without curriculum learning.
• nrehiew summarized the report's most distinctive infra practices: masking infra failures (e.g., unavailable tools) from trajectories; using the same harness within a group for normalization despite multiple harnesses; estimating HBM usage from rollout length.
• The setup forgoes PipelineRL-style KV-cache reuse in favor of re-prefilling after policy updates; stochasticchasm speculates that with only 30 steps and large per-step policy changes, re-prefilling may be worth it.
• stochasticchasm observed that the grader consumed substantial compute and wondered which model served as grader; agent judging may become a new scaling axis.
• On MOPD2 (report point 6), tokenbender speculates it was likely completed earlier, not part of this RL replay/live run, and considers expert-teacher design more effective than large-scale mixed RL.

Unconfirmed

• The "1M-context RL training" claim is stochasticchasm's inference from the technical material, phrased as suspected.
• Critics note the absence of batch-size ablations, so whether large batches substitute for larger group sizes or a critic remains conjecture; MOPD2's timing is also tokenbender's speculation.

Why it matters

• If 1M-context RL training is confirmed, it marks a new breakthrough on the scaling axis, potentially benefiting long-context agent tasks.
• Approaching frontier performance in 30 steps, plus 13% policy drift and no need for curriculum learning, suggests large-batch RL sample efficiency may far exceed prior assumptions.
• The GRPO stabilization checklist, trajectory cleaning and harness normalization tricks, high grader compute share, and re-prefill tradeoffs offer a rare first-hand reference for understanding the cost structure and infrastructure design of large-scale RL.

2026-09-22 ~ 2026-09-23 · 15 related posts

Full story(3 episodes)→

• Episode 1: Xiaomi MiMo RL Training Details Spark Debate: Suspected 1M-Context RL, Big Gains in Just 30 Steps(2026-09-22, 15 posts)
• Episode 2: Xiaomi's MiMo-V2.6 Bets on Reinforcement Learning Scaling Across 1000+ TPUs(2026-09-22, 10 posts)
• Episode 3: Xiaomi's MoE RL stability paper: router-aware GRPO tweaks explained(2026-09-22, 2 posts)

• Just 30 RL Steps Drive Dramatic Model Gains With No Plateau in Sight — stochasticchasm · 2026-09-22
• Classic MiMo architecture with huge RL batches: 1568 tasks x 16 rollouts per step, surprising results — stochasticchasm · 2026-09-22
• [source] Xiaomi MiMo may be first RL at 1M context, with agent-as-a-judge as new scaling axis — stochasticchasm · 2026-09-22
• Researchers debate GRPO at scale: standard formulation, missing batch-size ablations — stochasticchasm · 2026-09-22
• Standard GRPO at 1M scale: why no critic models, and what counts as "behaviors"? — stochasticchasm · 2026-09-22
• RL infra detail: re-prefill over PipelineRL's KV cache reuse, batch size for GPU utilization — stochasticchasm · 2026-09-22
• RL training detail: re-prefilling instead of PipelineRL's cached KV, with batch size framed as a GPU-utilization lever — stochasticchasm · 2026-09-22
• [source] RL run spends a lot of compute on graders; is re-prefilling worth it over 30 steps — stochasticchasm · 2026-09-22
• Xiaomi's model jumps near the frontier with just 30 RL steps, no plateau in sight — teortaxesTex · 2026-09-22
• 13% Policy Drift Staleness Suggests RL Envs Needed No Curriculum, Tokenbender Argues — tokenbender · 2026-09-22
• [source] RL training stability tricks: prompt-mean loss, asymmetric clipping and entropy-adaptive GRPO — tokenbender · 2026-09-22
• Reading Xiaomi's MoE RL Release: Specialist Teachers Beat Big Mixed RL on Hard Tasks — tokenbender · 2026-09-22
• nrehiew_ dumps the unique RL training infra tricks: trajectory masking to HBM estimation — nrehiew_ · 2026-09-23
• MiMo-V2.6 Improved Dramatically in Just 30 RL Steps, No Plateau in Sight — burny_tech · 2026-09-23

1 near-duplicate retellings: stochasticchasm