Agentic RL data scaling laws paper: environments beat tasks and trajectories, 1.7x compute savings
omarsar0 · x · 2026-10-20
A paper from MBZUAI, McGill and collaborators decomposes agentic RL data into three axes — tasks, environments, trajectories — and fits per-axis scaling laws for LLMs across math/code, SWE, web and UI/tool domains.
Key findings
- Across model sizes, families, domains, and both PPO and GRPO, scaling exponents order the same way: environments > tasks > trajectories.
- The environment advantage comes mainly from world diversity (genuinely different settings); varying structure within one world helps about as much as adding tasks.
- Each synthesis method gets a discount factor and a critical synthetic-to-real ratio where training collapses; these ratios differ by an order of magnitude across axes.
- Budgeting by their Chinchilla-style recipe yields up to 1.7x compute efficiency over the dominant scale-the-trajectories practice, validated at held-out scale.
More from Research
- PretrainGym pits frontier agents on pretraining data curation: Kimi K3 wins at 1.042 — xeophon · 2026-10-20
- Claude autonomously generates 318,000 protein designs in 64 hours, hits 14 of 15 targets — kimmonismus · 2026-10-20
- Claude runs 64-hour autonomous research: 318k protein designs, 17 binders validated — kimmonismus · 2026-10-20
- ACEdit Stabilizes 15-Turn Image Editing With Asymmetric Conditioning — tau · 2026-10-20
- Salesforce Trains Reasoning Models to Batch-Generate Diverse Research Ideas — Salesforce · 2026-10-20
- LightOn's blog details SlideOn's method, results and ablations — MoonL88537 · 2026-10-20