Paper finds pretraining loss predicts post-RL reasoning gains, using chess and math tests
burny_tech · x · 2026-07-22
Pretraining loss predicts how much RL helps reasoning models
A paper titled Understanding Reasoning from Pretraining to Post-Training argues that the benefits of RL for reasoning are largely determined by pretraining.
- The authors build a controlled LLM-style pipeline using chess as a testbed: pretraining, SFT on synthetic reasoning traces, then GRPO on verifiable puzzles.
- They find that post-RL performance at a given compute level is well predicted by pretraining loss.
- More pretraining tokens also make RL improvements arrive faster.
- RL does more than sharpen easy cases: it can also uncover correct moves that were hidden after SFT on harder puzzles.
- The same pattern appears in a math experiment with a 1B model: longer-pretrained checkpoints reach higher post-RL performance and improve faster under RL.
The paper’s main claim is that pretraining and RL are tightly coupled, so the pretraining stage strongly shapes how much post-training compute can buy you.
More from Research
- CHI 2026 best paper uses EMS and embodied AI to guide physical tasks — MacrinePhD · 2026-07-22
- A closer look at whether safe reasoning training can avoid RL pathologies — xuanalogue · 2026-07-22
- See2Act teaches robots where to look and how to act in one denoising loop — heghbalz · 2026-07-22
- Filtering Reasoning Traces for Alignment Before SFT to Avoid RL Pathologies — xuanalogue · 2026-07-22
- U.S. labs lean on RL while Chinese labs favor SFT on successful traces — goyalshaliniuk · 2026-07-22
- Paper Reader lets users explain passages, formulas and citations inside research papers — tumanian · 2026-07-22