RL pathologies stem from online training, not reward design, researcher argues
QuintinPope5 · x · 2026-10-13
Quintin Pope argues that the convention of pairing RL with online training (e.g., RLVR: sample trajectory, score, train on it) is the main source of most pathologies—more than the specifics of the reward function. He frames RL as a generalization of imitation learning where trajectory rewards control imitation strength, including "negative imitation," pushing back on RL's icky reward-hacking reputation.
More from Research
- AgentSWE benchmark: no coding agent comes close to delivering the agents it's asked to build — Jiahao Wang · 2026-10-13
- Schema agent hits ~99% RHAE on ARC-AGI-3 public set, full paper and code released — wzenus · 2026-10-13
- World model research: more data won't suffice—squeeze more from it; even intuitive physics is still out of reach — mangahomanga · 2026-10-13
- Indie dev builds own serving library to train SVG model, finds evolution loses to SGD — cephaloform · 2026-10-13
- Harvard & NTT Research: cooperating AI agent populations hit an Allee-effect takeoff threshold — Hidenori8Tanaka · 2026-10-13
- RT-Super trains tumor segmentation from radiology reports, lifting Dice from 16 to 30 without masks — maier_ak · 2026-10-13