Paper finds pretraining loss predicts post-RL reasoning gains, using chess and math tests
burny_tech · x · 2026-07-22
Pretraining loss predicts how much RL helps reasoning models
A paper titled Understanding Reasoning from Pretraining to Post-Training argues that the benefits of RL for reasoning are largely determined by pretraining.
- The authors build a controlled LLM-style pipeline using chess as a testbed: pretraining, SFT on synthetic reasoning traces, then GRPO on verifiable puzzles.
- They find that post-RL performance at a given compute level is well predicted by pretraining loss.
- More pretraining tokens also make RL improvements arrive faster.
- RL does more than sharpen easy cases: it can also uncover correct moves that were hidden after SFT on harder puzzles.
- The same pattern appears in a math experiment with a 1B model: longer-pretrained checkpoints reach higher post-RL performance and improve faster under RL.
The paper’s main claim is that pretraining and RL are tightly coupled, so the pretraining stage strongly shapes how much post-training compute can buy you.
More from Research
- Token Collapse: Study Finds Graph Soft Prompting Tokens in LLMs Fail to Encode Structure — Sapienza · 2026-10-27
- UniAction: unified action conditioning lets one world model handle mixed action types — ZeYanjie · 2026-10-27
- Infini-AI Lab open-sources Hermit, a framework for agent harness evolution and continual learning — BeidiChen · 2026-10-27
- Meta FAIR's Self-Direction turns agent failures into reusable LoRA skills, +13.8pp on Terminal-Bench — karen_ullrich · 2026-10-27
- DONIC hits 86.79% on bimanual dexterous tracking, crushing prior SOTA's 9.43% — liuziwei7 · 2026-10-27
- Why don't more LLMs ship 10M-token context? A researcher asks about attention tradeoffs — agarwl_ · 2026-10-27