UIUC paper: Fisher-selected 0.1% of parameters recovers 85% of full RL training gains
burny_tech · x · 2026-10-20
A University of Illinois paper explains why LLM post-training (RL, on-policy distillation, near-policy SFT) produces highly sparse parameter updates.
- Theory: the diagonal Fisher information measures each parameter's sensitivity in the output distribution, independent of any reward or teacher signal. The authors show small diagonal Fisher implies small expected gradients across training objectives, giving a unified explanation for sparsity.
- Empirics: fixed sparse masks selected from initial Fisher before training retain most of full-training gains — on Qwen3-1.7B, updating just 0.1% of parameters chosen by Fisher recovers 85.2% of full RL improvement, and 5% recovers 98.8%.
- Mechanism: low Fisher in on-policy training arises because high-probability next tokens tend to have similar parameter sensitivities.
The work positions diagonal Fisher as a unifying lens linking update sparsity to on-policy training dynamics.
More from Research
- Logarithmic returns come from opportunities spread across orders of magnitude of cost — DimitrisPapail · 2026-10-20
- Emergent curriculum without labels: prompt weights diverge 1.6x vs 0.75x over training — shangbinfeng · 2026-10-20
- Stackelberg Alignment turns LLM disagreement into a training curriculum via bandit leader — shangbinfeng · 2026-10-20
- RL fine-tuning Qwen3.5-0.8B for under $20 on HF Jobs lifts pass@1 from 10% to 36% — LysandreJik · 2026-10-20
- Two New Robotics Benchmarks Released: MicroPlanBench and MicroAnnotationBench First Rankings — chris_j_paxton · 2026-10-20
- ICLR reviewer despairs: 5 assigned papers nearly all AI-written, calls peer review broken — peterbhase · 2026-10-20