New Research Proposes Joint Scaling Law for Pretraining and RL

A study titled "Understanding Reasoning from Pretraining to Post-Training" introduces a controlled chess testbed, unifying pretraining, SFT, and reinforcement learning (RL) into a computable framework. It provides quantitative answers to long-standing questions about compute allocation and how RL changes models. Paper, code, and models are open-sourced.

Key Details and Core Laws

In 36 experiments spanning 20M to 1B model sizes, the team found that optimal compute allocation shifts with budget: prioritize pretraining at low compute, but as model size grows from 50M to 700M, the optimal strategy leans toward RL, with RL compute reaching up to ~30%. Pretraining tokens still roughly follow Chinchilla scaling. The authors propose a joint scaling law: at high compute, RL performance is well predicted by pretraining loss, and more pretraining tokens strengthen this relationship.

RL Mechanism and Generalization

Regarding RL's mechanism, the authors note that on simple tasks, RL mainly sharpens already decent strategies, while on harder tasks it can uncover low-probability tail solutions. This means RL is not limited to fine-tuning but can discover rare reasoning patterns. @burny_tech had a similar industry question about whether RLVR reinforces existing abilities or elicits new reasoning; this study partially answers it. The chess findings extend to OLMo's natural language math reasoning, confirming that pretraining loss strongly predicts RL gains.

2026-07-20 ~ 2026-07-21 · 18 related posts

• Does RLVR Reinforce Existing Skills or Spark New Reasoning? — burny_tech · 2026-07-20
• Paper: Evolution of Reasoning from Pre-training to Post-training — Jingyan Shen · 2026-07-20
• Joint pretraining-RL scaling says RL share should rise with compute — teortaxesTex · 2026-07-20
• New chess study fits a pretraining–RL scaling law to optimize compute allocation — micahgoldblum · 2026-07-21
• [source] New paper finds a joint scaling law from pretraining through post-training — Pavel_Izmailov · 2026-07-21
• A chess testbed studies how to split compute between pretraining, SFT, and RL — Pavel_Izmailov · 2026-07-21
• [source] Chess scaling testbed finds more pretraining improves RL pass@1 and final performance — Pavel_Izmailov · 2026-07-21
• A joint scaling law links RL performance to model size, pretraining tokens, and compute — Pavel_Izmailov · 2026-07-21
• More compute shifts the optimum from pretraining toward RL in a chess scaling law — Pavel_Izmailov · 2026-07-21
• RL mostly sharpens easy tasks, but can find tail solutions on harder ones — Pavel_Izmailov · 2026-07-21
• [source] OLMo results show pretraining loss still predicts RL gains on math reasoning — Pavel_Izmailov · 2026-07-21
• The chess testbed is meant to probe basic learning and scaling questions in academia — Pavel_Izmailov · 2026-07-21
• Chess scaling testbed paper, code and models are now available — Pavel_Izmailov · 2026-07-21
• Another share of the paper on unified pretraining and RL scaling laws — Pavel_Izmailov · 2026-07-21
• Pretrain-RL study finds the optimal RL share rises from 20% to 30% as models scale — Pavel_Izmailov · 2026-07-21
• New paper traces LLM reasoning from pretraining to post-training — gerardsans · 2026-07-21
• New notes on a paper argue pretraining determines how far RL can still improve a model — tokenbender · 2026-07-21
• Study finds pretraining loss can predict RL scaling before saturation — heghbalz · 2026-07-21