A 1T-parameter MoE model reportedly learned math reasoning with zero-RL and no human solutions

i_dg23 · x · 2026-07-27

A new paper claims a 1T-parameter MoE model learned math reasoning with zero-RL, without showing any human-written solution traces.

Setup

Base model: Ling-2.5-1T-Base, a 1T-parameter MoE where only about 63B parameters are active per token.
Training used a four-stage pipeline:
First-stage RL to amplify low-probability reasoning tokens and surface latent reasoning.
Self-distillation to compress long chains of thought and reset train/inference drift between Megatron and SGLang.
Second-stage RL for stability and sustained improvement.
Third-stage RL with adaptive depth, allowing about 4,000 tokens for easy problems and up to 64,000 tokens for hard ones.
Infrastructure: 320 H200 GPUs.

Reported results

After stage 1, the model already reached 84.2% on AIME 2026.
The final model reportedly won pairwise CoT comparisons against MiniMax-M2.7 at 78% and GLM-5.1 at 76%.
It used 6,368 tokens to solve tasks, versus 17,220 for GLM-5.1 and 14,115 for Kimi-K2.6.
The distillation set was only 100k samples, compared with 800k for DeepSeek-R1.

The image also highlights infrastructure optimizations such as mixed-precision control and context-parallel optimization, plus emergent behaviors like anthropomorphism, structured format, parallel reasoning, and context anxiety.

Original post →

A 1T-parameter MoE model reportedly learned math reasoning with zero-RL and no human solutions

More from Research