RL Gains Concentrate on Easy Questions; 'Never Give Up' Resampling Tackles the Hard Ones
teortaxesTex · x · 2026-09-16
Researchers led by Nat Lambert find that RL gains on LLMs concentrate mostly on easy questions — a phenomenon they call the Matthew Effect for RL. Their fix is simple: when a GRPO group contains only wrong completions (zero gradient), resample with 0.9 probability until finding batches with nonzero gradient. Called "Never Give Up", the technique pairs with async RL to allocate more compute to harder problems, and it works. Paper and blog linked.
Related event: Research Reveals RL Matthew Effect; NGU Sampling Targets Hard Problems(4 posts)→
More from Research
- How to Loop Your Transformer: Meta/MIT arXiv paper on loop scaling laws — rohanpaul_ai · 2026-11-02
- Meta & MIT paper: looping transformer layers beats adding depth at fixed size — rohanpaul_ai · 2026-11-02
- AI tell-tale word 'delve' declining in latest NeurIPS papers, analysis shows — JungWooHa2 · 2026-11-02
- ByteDance Seed's Loop-μP paper draws 'Chinese loopmaxxers' buzz — scaling01 · 2026-11-02
- New benchmark shows frontier VLMs still can't read Braille despite 43M blind users — prajdabre · 2026-11-02
- Diffusion models emerge from associative memory: spurious states may signal generalization — DimaKrotov · 2026-11-02