Paper finds pretraining loss predicts post-RL reasoning gains, using chess and math tests
burny_tech · x · 2026-07-22
Pretraining loss predicts how much RL helps reasoning models
A paper titled Understanding Reasoning from Pretraining to Post-Training argues that the benefits of RL for reasoning are largely determined by pretraining.
- The authors build a controlled LLM-style pipeline using chess as a testbed: pretraining, SFT on synthetic reasoning traces, then GRPO on verifiable puzzles.
- They find that post-RL performance at a given compute level is well predicted by pretraining loss.
- More pretraining tokens also make RL improvements arrive faster.
- RL does more than sharpen easy cases: it can also uncover correct moves that were hidden after SFT on harder puzzles.
- The same pattern appears in a math experiment with a 1B model: longer-pretrained checkpoints reach higher post-RL performance and improve faster under RL.
The paper’s main claim is that pretraining and RL are tightly coupled, so the pretraining stage strongly shapes how much post-training compute can buy you.
More from Research
- Causal-only attention for non-generative tasks is wasteful, argues HF engineer — antoine_chaffin · 2026-09-11
- Catholic University of Chile researcher: scaling AI feedback is key to sustainable medical education — julianvarascom · 2026-09-11
- Nature paper images cellular activity across all organs, revealing body-wide circuits — arjunrajlab · 2026-09-11
- SignNet 1M Dataset Released for Sign Language Research — ducha_aiki · 2026-09-11
- ECCV26 Oral: Flow Matching Enables Single-Stage Multi-View Point Cloud Registration — ducha_aiki · 2026-09-11
- InFlux++ Method Released — ducha_aiki · 2026-09-11