Inside Xiaomi MiMo's RL Scaling Playbook

Episode 1 · Xiaomi MiMo RL Training Details Spark Debate: Suspected 1M-Context RL, Big Gains in Just 30 Steps (2026-09-22, 15 posts)

Xiaomi MiMo-V2.6's RL training details triggered intense discussion in overseas technical communities. stochasticchasm suspects this is the industry's first RL training on a 1M-token context, with dramatic gains in only 30 steps and no plateau on the training curve; tokenbender and nrehiew offered systematic breakdowns of the algorithmic modifications, training efficiency, and infra practices. The converging takeaway: large-batch, stabilized standard GRPO may deliver far higher sample efficiency than previously expected.

Confirmed

• MiMo uses a classic architecture, described by stochasticchasm as "simple and strong".
• Extremely large per-step batches: 1568 tasks × 16 rollouts per task, consuming enormous token counts per run.
• stochasticchasm noted the model improved dramatically in only 30 RL steps, far fewer than expected, with no plateau; he and commenter nrehiew questioned why training was not continued.
• Training uses a very standard GRPO form without a critic model. stochasticchasm guesses the large batch already suffices to reduce gradient noise, even without larger group sizes.
• tokenbender catalogued targeted GRPO modifications: prompt-mean loss over token-mean loss (DAPO) to avoid overweighting long answers; separate clipping bounds for positive/negative advantages. The philosophy: keep the algorithm simple, modify for stability and scale.
• tokenbender argued that high throughput, heterogeneous environments, and only 13% policy drift staleness imply most environments run or improve directly on base weights, without curriculum learning.
• nrehiew summarized the report's most distinctive infra practices: masking infra failures (e.g., unavailable tools) from trajectories; using the same harness within a group for normalization despite multiple harnesses; estimating HBM usage from rollout length.
• The setup forgoes PipelineRL-style KV-cache reuse in favor of re-prefilling after policy updates; stochasticchasm speculates that with only 30 steps and large per-step policy changes, re-prefilling may be worth it.
• stochasticchasm observed that the grader consumed substantial compute and wondered which model served as grader; agent judging may become a new scaling axis.
• On MOPD2 (report point 6), tokenbender speculates it was likely completed earlier, not part of this RL replay/live run, and considers expert-teacher design more effective than large-scale mixed RL.

Unconfirmed

• The "1M-context RL training" claim is stochasticchasm's inference from the technical material, phrased as suspected.
• Critics note the absence of batch-size ablations, so whether large batches substitute for larger group sizes or a critic remains conjecture; MOPD2's timing is also tokenbender's speculation.

Why it matters

• If 1M-context RL training is confirmed, it marks a new breakthrough on the scaling axis, potentially benefiting long-context agent tasks.
• Approaching frontier performance in 30 steps, plus 13% policy drift and no need for curriculum learning, suggests large-batch RL sample efficiency may far exceed prior assumptions.
• The GRPO stabilization checklist, trajectory cleaning and harness normalization tricks, high grader compute share, and re-prefill tradeoffs offer a rare first-hand reference for understanding the cost structure and infrastructure design of large-scale RL.

• Just 30 RL Steps Drive Dramatic Model Gains With No Plateau in Sight — stochasticchasm · 2026-09-22
• Model improved dramatically in just 30 RL steps with huge batch sizes — stochasticchasm · 2026-09-22
• Classic MiMo architecture with huge RL batches: 1568 tasks x 16 rollouts per step, surprising results — stochasticchasm · 2026-09-22
• Xiaomi MiMo may be first RL at 1M context, with agent-as-a-judge as new scaling axis — stochasticchasm · 2026-09-22
• Researchers debate GRPO at scale: standard formulation, missing batch-size ablations — stochasticchasm · 2026-09-22
• Standard GRPO at 1M scale: why no critic models, and what counts as "behaviors"? — stochasticchasm · 2026-09-22
• RL infra detail: re-prefill over PipelineRL's KV cache reuse, batch size for GPU utilization — stochasticchasm · 2026-09-22
• RL training detail: re-prefilling instead of PipelineRL's cached KV, with batch size framed as a GPU-utilization lever — stochasticchasm · 2026-09-22
• RL run spends a lot of compute on graders; is re-prefilling worth it over 30 steps — stochasticchasm · 2026-09-22
• Xiaomi's model jumps near the frontier with just 30 RL steps, no plateau in sight — teortaxesTex · 2026-09-22
• 13% Policy Drift Staleness Suggests RL Envs Needed No Curriculum, Tokenbender Argues — tokenbender · 2026-09-22
• RL training stability tricks: prompt-mean loss, asymmetric clipping and entropy-adaptive GRPO — tokenbender · 2026-09-22
• Reading Xiaomi's MoE RL Release: Specialist Teachers Beat Big Mixed RL on Hard Tasks — tokenbender · 2026-09-22
• nrehiew_ dumps the unique RL training infra tricks: trajectory masking to HBM estimation — nrehiew_ · 2026-09-23
• MiMo-V2.6 Improved Dramatically in Just 30 RL Steps, No Plateau in Sight — burny_tech · 2026-09-23

Episode 2 · Xiaomi's MiMo-V2.6 Bets on Reinforcement Learning Scaling Across 1000+ TPUs (2026-09-22, 10 posts)

Xiaomi released MiMo-V2.6, its largest reinforcement learning scaling effort to date, spanning coding, general reasoning, vision, automation and cybersecurity. The team completed full-parameter RL training on the 310B model across 1000+ TPUs, stably running 1000+ steps and processing 2.7-3.7 billion tokens per step. Luo Fuli's retrospective frames the core question: after years of pretraining scaling laws, can RL scale capability the same way—Xiaomi has been betting on this for the past half year.

Confirmed

• The paper explicitly scales three RL factors: batch size and throughput, environment diversity, and grader compute. Per @DrSingularity, each RL step processes about 1568 samples and 2.7-3.7 billion tokens, with context lengths up to 1 million tokens.
• The MiMo team disclosed JAX+TPU-based large-scale RL: full-parameter RL training of the 310B MiMo-V2.6 on 1000+ TPUs, stably running 1000+ steps; scaling up mainly requires config changes rather than code rewrites.
• @tokenbender's thread notes the large-batch justification stays at throughput and GPU-scaling convenience. Environment synthesis follows standard good practices—clear specs, sampling from rich real-world scenario pools, curating quality seed tasks with grounding. Section 4.2.5's multi-harness training earned his approval, as open-source users often build their own harnesses.
• The paper discusses CodeMidas's source-driven synthesis. Luo Fuli's retrospective mentioned RL scale details such as generating 25,000 agent trajectories per step.

Unconfirmed

• Long-horizon tasks appear to rely mainly on agent-driven synthesis; tokenbender is skeptical of their effectiveness and plans to follow up.

Why it matters

• tokenbender observes MiMo's tasks are grounded from diverse sources rather than built from scratch—pure self-play is not a substitute; since the tasks resemble daily workflows, he expects MiMo v2.6 could replace Sol for daily work.
• Running 310B full-parameter RL on JAX+TPU with config-only scaling offers the open-source community an engineering reference for large-scale RL training.

• Full-Parameter RL on TPUs: peano_ai Runs 310B MiMo-V2.6 Across 1,000+ TPUs — simonguozirui · 2026-09-22
• Xiaomi MiMo runs full-parameter RL on 310B model across 1000+ TPUs with JAX — AccBalanced · 2026-09-22
• Xiaomi MiMo v2.6 ships with RL as the hero, scaling batch size, env diversity and grader compute — tokenbender · 2026-09-22
• Thread recap: MiMo v2.6's environment scaling principles grounded in diverse real sources — tokenbender · 2026-09-22
• MiMo leans on CodeMidas-style source-driven synthesis and agent-driven long-horizon tasks — tokenbender · 2026-09-22
• MiMo v2.6 details: grounded environment synthesis and multi-harness training for open source users — tokenbender · 2026-09-22
• Dev breaks down paper: multi-harness training and environment synthesis principles — tokenbender · 2026-09-22
• Xiaomi MiMo v2.6 ships: a deep dive into its RL scaling of batch size, environments, graders — tokenbender · 2026-09-22
• Xiaomi's Luo Fuli Recaps MiMo-V2.6 RL Run: 25K Agent Trajectories Per Step — bookwormengr · 2026-09-22
• Xiaomi MiMo-V2.6 scales RL to 3.7B tokens per step, targeting self-improvement — Dr_Singularity · 2026-09-23

Episode 3 · Xiaomi's MoE RL stability paper: router-aware GRPO tweaks explained (2026-09-22, 2 posts)

tokenbender breaks down Xiaomi's arXiv paper on stable MoE RL: rather than replacing GRPO, the team adds targeted fixes such as router-aware importance-sampling rescaling and prompt-mean loss for stability and scale.

• Sticking with GRPO: The Four Targeted Mods Behind Xiaomi's Stable MoE RL — tokenbender · 2026-09-22
• New arXiv Paper: Router-Aware Importance Sampling Stabilizes MoE RL Training — tokenbender · 2026-09-22