New Research Proposes Joint Scaling Law for Pretraining and RL

A new study titled "Understanding Reasoning from Pretraining to Post-Training" introduces a controlled chess testbed to integrate the pretraining, SFT, and reinforcement learning (RL) of large models into a unified computable framework. This work provides quantitative answers to long-standing questions about how compute should be allocated between pretraining and post-training, and how RL actually alters models. The paper, code, and models have been open-sourced.

Key Details and Core Scaling Laws

Through 36 combinatorial experiments covering model sizes from 20M to 1B, the research team found that the optimal compute allocation shifts with the budget. At lower compute levels, priority should be given to pretraining. However, as models scale from 50M to 700M and the total budget increases, the optimal strategy gradually shifts towards RL, with the optimal RL compute proportion rising to about 30%. Furthermore, the number of pretraining tokens generally follows the Chinchilla scaling law. The authors also propose a joint scaling law: in high-compute regions, RL performance can be well predicted by pretraining loss, and this predictive relationship strengthens as pretraining tokens increase.

RL Mechanisms and Generalization

Regarding how RL works, the authors note that for simple tasks, RL primarily sharpens already decent strategies (sharpening the distribution). For harder tasks, it can sometimes discover tail-end solutions with very low probability in the original strategy. This means RL is not merely fine-tuning but can unearth rare reasoning patterns. @burnytech previously raised a similar industry question about whether RLVR reinforces existing capabilities or elicits new reasoning, which this study partially answers. Additionally, the authors extended their chess testbed conclusions to natural language math reasoning tasks using OLMo, verifying that the strong predictive power of pretraining loss on RL gains still holds.

2026-07-20 ~ 2026-07-21 · 18 related posts

• Does RLVR Reinforce Existing Skills or Spark New Reasoning? — burny_tech · 2026-07-20
• Paper: Evolution of Reasoning from Pre-training to Post-training — Jingyan Shen · 2026-07-20
• Joint pretraining-RL scaling says RL share should rise with compute — teortaxesTex · 2026-07-20
• New chess study fits a pretraining–RL scaling law to optimize compute allocation — micahgoldblum · 2026-07-21
• [source] New paper finds a joint scaling law from pretraining through post-training — Pavel_Izmailov · 2026-07-21
• [source] A chess testbed studies how to split compute between pretraining, SFT, and RL — Pavel_Izmailov · 2026-07-21
• Chess scaling testbed finds more pretraining improves RL pass@1 and final performance — Pavel_Izmailov · 2026-07-21
• A joint scaling law links RL performance to model size, pretraining tokens, and compute — Pavel_Izmailov · 2026-07-21
• More compute shifts the optimum from pretraining toward RL in a chess scaling law — Pavel_Izmailov · 2026-07-21
• [source] RL mostly sharpens easy tasks, but can find tail solutions on harder ones — Pavel_Izmailov · 2026-07-21
• OLMo results show pretraining loss still predicts RL gains on math reasoning — Pavel_Izmailov · 2026-07-21
• The chess testbed is meant to probe basic learning and scaling questions in academia — Pavel_Izmailov · 2026-07-21
• Chess scaling testbed paper, code and models are now available — Pavel_Izmailov · 2026-07-21
• Another share of the paper on unified pretraining and RL scaling laws — Pavel_Izmailov · 2026-07-21
• Pretrain-RL study finds the optimal RL share rises from 20% to 30% as models scale — Pavel_Izmailov · 2026-07-21
• New paper traces LLM reasoning from pretraining to post-training — gerardsans · 2026-07-21
• New notes on a paper argue pretraining determines how far RL can still improve a model — tokenbender · 2026-07-21
• Study finds pretraining loss can predict RL scaling before saturation — heghbalz · 2026-07-21