Does RL Expand LLM Reasoning or Just Reweight It? New Study Offers Theory and Scaling Laws
burny_tech · x · 2026-10-28
- A new study revisits conflicting RL evidence across the Qwen and Gemma families: math training yields cross-domain gains, yet Pass@1 improvements can coexist with lower Pass@N than the base model.
- Trace analysis shows RL shifts the distribution over reasoning strategies. The authors propose a two-stage autoregressive policy model separating strategy selection from execution, proving how RL's implicit bias reshapes strategy preferences—boosting some tasks while suppressing strategies others need.
- This explains transfer/forgetting and coverage changes at finite sampling budgets, plus theoretical justifications for log-sigmoid and log-linear RL compute scaling laws, validated empirically.
Related event: Berkeley Paper: RL Reweights Rather Than Expands LLM Reasoning Boundary(3 posts)→
More from Research
- DFR Framework Tackles Reward Fine-Tuning in Diffusion Models Without Losing Diversity — mmbronstein · 2026-10-28
- alphaXiv ships Mobile 2.0 with AI paper reader and one-click visual blogs — datawithsuman · 2026-10-28
- Researchers screened 175 Spanish books ahead of time and found multiple AI-generated positives — TuhinChakr · 2026-10-28
- Xiaohongshu and Fudan Propose DR-Rubric: Agent-Written Rubrics as RL Rewards — jiqizhixin · 2026-10-28
- Ben Goertzel: AI Peer Review Already Beats Humans, and Human Bias Is Harder to Fix — bengoertzel · 2026-10-28
- Developer Joins DeepMind's Antigravity Science Program, Claims Gemini Argon Is Frontier in Biomedicine — DeryaTR_ · 2026-10-28