Study: Training a Single Layer Can Match Full-Parameter RL
机器之心 · wechat · 2026-07-08
A joint team from the University of Minnesota, Peking University, and Amazon challenged the implicit assumption that RL post-training should uniformly update all Transformer layers. Introducing a "layer contribution" metric, they discovered that RL gains are highly concentrated in a few intermediate layers. On Qwen3-1.7B, training only the 10th layer recovers 114% of the gains from full-parameter RL, while the weakest layer recovers only 28%, showing a 4x difference between the best and worst layers.
Experiments spanned 7 models, 2 model families, 3 RL algorithms (GRPO/Dr.GRPO/GiGPO), and 3 task types: math reasoning, code generation, and agentic decision-making. The paper demonstrates that single-layer training strategies based on these findings can consistently outperform standard full-parameter RL training, offering new insights for RL post-training algorithm design.
Related event: Training Single Transformer Layer Can Match Full-Parameter RL, Study Finds(2 posts)→
More from Research
- Nature paper images cellular activity across all organs, revealing body-wide circuits — arjunrajlab · 2026-09-11
- SignNet 1M Dataset Released for Sign Language Research — ducha_aiki · 2026-09-11
- ECCV26 Oral: Flow Matching Enables Single-Stage Multi-View Point Cloud Registration — ducha_aiki · 2026-09-11
- InFlux++ Method Released — ducha_aiki · 2026-09-11
- Skyfall GS Uses Flux to Refine Gaussian Splatting, Accepted at ECCV 2026 — ducha_aiki · 2026-09-11
- Could 10k agents discover learning methods beyond backprop, or just tweak existing ones? — SeunghyunSEO7 · 2026-09-11