Microsoft Research Proposes RLTR: Rewarding Transferable Reasoning Steps to Fix Fragile RL Chains
burkov · x · 2026-09-30
Current RL with verifiable rewards (RLVR) trains LLMs by checking only whether the final answer is correct, not intermediate steps. This cuts reward hacking and annotation cost but creates a reliability problem: models often reach correct answers via fragile, idiosyncratic reasoning paths by luck, so consensus across multiple sampled paths degrades performance and consistency.
This Microsoft Research paper introduces and evaluates Reinforcement Learning with Transferable Reward (RLTR), a framework that rewards a model when its partial reasoning can be successfully continued—making intermediate steps robust and reusable, improving consistency across samples.
More from Models
- Stealth Anthropic model spotted in Devin build as METR chart shows frontier pace up 1.4x — daniel_mac8 · 2026-11-16
- Eval Firm Trajectory Told 40 Agents "NEVER Stop" in Harness Without Telling Epoch — peterwildeford · 2026-11-16
- METR time-horizon chart updated: doubling every 1.7 months since Opus 4.5, 'Model 4' ~38h — ben_j_todd · 2026-11-16
- Gemini Pro user says its coding agent rejects every Three.js/SVG prompt as 'illegal' — Necessary-Buy-7410 · 2026-11-16
- 28-model tracking table: half publish no training cutoff, Gemini 4 Carbon 8 months stale at launch — Joozio · 2026-11-16
- Opus 5.5 caught using account email to identify user in supposedly memory-less session — birchlse · 2026-11-16