UC Berkeley paper: RL reweights LLM reasoning strategies rather than expanding them
burny_tech · x · 2026-10-27
A UC Berkeley paper asks whether RL teaches LLMs new reasoning skills or merely reweights existing ones.
Key findings:
- Across Qwen and Gemma families, RL brings cross-domain gains and forgetting simultaneously; Pass@1 gains can coexist with lower Pass@N.
- RL mainly shifts which reasoning strategies the model prefers: on Qwen3-14B, RL reduces strategy diversity by 18% while lifting BoxNet Pass@1024 from 34% to 51%—fewer strategies can yield broader coverage.
- The authors build a two-stage autoregressive policy model separating strategy selection from execution, proving how RL's implicit bias reshapes strategy preferences, boosting some tasks while suppressing strategies needed by others.
- They also derive and validate log-sigmoid and log-linear scaling laws for RL compute.
Takeaway: RL largely reweights the existing reasoning space rather than expanding the boundary, reconciling previously conflicting evidence.
More from Models
- Speculation: xAI's announcement is likely Index, not Fable 5.5 — ChrisGPT · 2026-10-27
- AMD and vLLM team release open 27B multimodal decision model d3-ultra, #2 on Jev Decision Index — vllm_project · 2026-10-27
- Reasoning is signed: building a universal LLM gateway hits encrypted CoT walls — respan-81 · 2026-10-27
- Gemini 4 Argon reportedly beats GPT-6 Astra on a new CS research benchmark — Saboo_Shubham_ · 2026-10-27
- LlamaIndex releases OCRia-1, a 4B document parsing VLM topping ParseBench at 87.4 — llama_index · 2026-10-27
- Baidu open-sources Unlimited OCR 2: near-constant latency on documents of any length — Baidu_Inc · 2026-10-27