Proposal: replace online RL with imitation on verified successful trajectories
QuintinPope5 · x · 2026-10-13
Quintin Pope argues that most pathologies of RLVR-style methods come from the online training loop rather than the reward function details. He proposes "Self Imitation from Verified Completions": collect trajectories, verify task completion, then run pretraining-style imitation learning only on successful trajectories—an offline middle path between RL and imitation. Speculative, untested idea.
More from Research
- First formal theory of how mid-training shapes RL post-training in LLMs — burkov · 2026-10-13
- Andrew Wilson: scientific success hinges on asking questions, not raw problem-solving — andrewgwils · 2026-10-13
- Gene-editing pioneer: an LLM will zero-shot design clinic-grade editors within years — chaitjo · 2026-10-13
- Columbia study: MFA-trained writers prefer Claude and ChatGPT stories in blind tests — TuhinChakr · 2026-10-13
- Leading Chinese mathematicians unite behind AI research: 8 institutions, 1,000+ scientists — pradeepviswav · 2026-10-13
- 4DCodeBench: strong static reconstruction doesn't transfer to dynamic scenes — jwt0625 · 2026-10-13