Study: Training a Single Layer Can Match Full-Parameter RL
机器之心 · wechat · 2026-07-08
A joint team from the University of Minnesota, Peking University, and Amazon challenged the implicit assumption that RL post-training should uniformly update all Transformer layers. Introducing a "layer contribution" metric, they discovered that RL gains are highly concentrated in a few intermediate layers. On Qwen3-1.7B, training only the 10th layer recovers 114% of the gains from full-parameter RL, while the weakest layer recovers only 28%, showing a 4x difference between the best and worst layers.
Experiments spanned 7 models, 2 model families, 3 RL algorithms (GRPO/Dr.GRPO/GiGPO), and 3 task types: math reasoning, code generation, and agentic decision-making. The paper demonstrates that single-layer training strategies based on these findings can consistently outperform standard full-parameter RL training, offering new insights for RL post-training algorithm design.
Related event: Training Single Transformer Layer Can Match Full-Parameter RL, Study Finds(2 posts)→
More from Research
- Stanford Team Introduces Gigatoken, the World's Fastest Tokenizer — StanfordAILab · 2026-07-22
- Tabul AI launches Metal TreeSHAP to speed up Shapley values on Apple silicon — Scobleizer · 2026-07-22
- Reddit points to OpenAI’s ChatGPT Ads page — EcstaticAsparagus509 · 2026-07-22
- Open-source runtime lets each repo define its own AI code reviewer — ibabufrik · 2026-07-22
- DeepSWE: A New Benchmark for Evaluating AI Coding Agents on Real GitHub Issues — pmz · 2026-07-22
- A Rust space-economy sim runs hundreds of autonomous ships, built with Claude — kalcode · 2026-07-22