Matthew Effect in RL post-training: paper shows LLMs only get better at problems they can already solve
natolambert · x · 2026-09-16
A new paper by Michael Noukhovitch (shared by natolambert) reveals a Matthew Effect in RL post-training of LLMs: average eval curves hide that nearly all gains come from easy problems going from somewhat solved to mostly solved, while problems where the initial model scores 0 at pass@32 barely improve.
Key findings and approach:
- Analyzed RL training of Olmo 3.1 7B on AIME 2025, splitting 30 questions into easy/medium/hard by initial pass rates; the hardest bucket stayed near zero throughout
- Proposes Never Give Up, a method that keeps the model exploring initially unsolvable problems — intuitively scalable but non-trivial to make work in practice
- Comes with an interactive blog post and open-source code
Related event: Research Reveals RL Matthew Effect; NGU Sampling Targets Hard Problems(4 posts)→
More from Research
- ICML paper: finite context windows impose impossibility results for adversarial regret — burny_tech · 2026-11-02
- WASABI: Blei lab trains diffusion models to sample Boltzmann distributions with 3.3x fewer energy evaluations — skoularidou · 2026-11-02
- A complete math roadmap for machine learning: linear algebra, calculus, probability — TivadarDanka · 2026-11-02
- A base LLM plus 11 lines of code is all you need for a decision model — helloiamleonie · 2026-11-02
- Editor fully resigns from JMLR, arguing AI reviews beat rushed humans and journals need a rethink — roydanroy · 2026-11-02
- flowl: a probabilistic programming language built on optimal transport, not distributions — burny_tech · 2026-11-02