Researcher argues reward-free imitation of verified completions inherits most of RLVR's problems
QuintinPope5 · x · 2026-10-13
Quintin Pope proposes a hypothetical method, "Self Imitation from Verified Completions" (SIVC): like RLVR, it collects trajectories and verifies task completion, but instead of an explicit reward function, it runs pretraining-style imitation learning only on successful trajectories.
His claim: SIVC would inherit all of RLVR's problems. The leverage of a reward function in shaping models comes mainly from its ability to influence the future distribution of online-generated training data — and SIVC does exactly the same by filtering for successes, so the issue lies in that distribution shift, not in having an explicit reward.
Related event: Anthropic researcher frames pretraining as a degenerate case of RL(3 posts)→
More from Research
- First formal theory of how mid-training shapes RL post-training in LLMs — burkov · 2026-10-13
- Andrew Wilson: scientific success hinges on asking questions, not raw problem-solving — andrewgwils · 2026-10-13
- Gene-editing pioneer: an LLM will zero-shot design clinic-grade editors within years — chaitjo · 2026-10-13
- Columbia study: MFA-trained writers prefer Claude and ChatGPT stories in blind tests — TuhinChakr · 2026-10-13
- Leading Chinese mathematicians unite behind AI research: 8 institutions, 1,000+ scientists — pradeepviswav · 2026-10-13
- 4DCodeBench: strong static reconstruction doesn't transfer to dynamic scenes — jwt0625 · 2026-10-13