Study of 20 RL Models: Outcome RL Buys Finishing and Reliability, Rarely Reach
Shan Yang · hf · 2026-10-29
A single-author paper, "What Outcome RL Buys: Finishing, Reliability, Rarely Reach," compares 20 released RL models (1.5B–32B) against the base thinking models they started from, on physics and competition math under fixed token budgets.
- Core finding: outcome RL raises pass@1 but not pass@k at large k — it changes behavior rather than acquiring new problem-solving knowledge.
- Each pass@1 change is decomposed into Finishing (answering before the budget runs out), Reliability (being right once finished), and Reach (problems the base got wrong in all eight tries, net of its mirror).
- Results: where the base often hits the token cap, RL's gains come mainly from finishing faster; where it already finishes, from higher reliability; reach gains are minimal at fixed budgets. Grouped bar charts quantify that finishing and reliability account for nearly all of the pass@1 improvement.
More from Models
- Gemini 4 Carbon benchmarks prompt TheZvi to declare Gemini 4 Argon 'was' near-frontier — TheZvi · 2026-10-29
- Preprint: Human peer reviewers caught only 27% of major errors, all 15 AI models beat them (53-96%) — anshulkundaje · 2026-10-29
- Study: top reasoning effort costs up to 42x more on tasks low effort already solves — zainhas · 2026-10-29
- Giotto launches as 'European sovereign AI,' gets exposed as Kimi-based with benchmark EULA gag — TheZachMueller · 2026-10-29
- Google Announces Gemini 4 Carbon, Its Next Frontier Model, Starting With Cyber Defenders — Snoo26837 · 2026-10-29
- Suhail launches Particle: particle-1 matches performance with ~45% fewer tokens — ycombinator · 2026-10-29