EEPO: Adaptive Sample-Then-Forget Rollouts Break Diversity Collapse in RLVR Training
burkov · x · 2026-11-13
- In RLVR training (reinforcement learning with verifiable rewards), LLMs solving math/logic problems tend to overexploit high-probability solution paths, causing output diversity to collapse early in training and trapping models in local optima with poor out-of-distribution generalization.
- The article introduces EEPO (Exploration-Enhanced Policy Optimization), an adaptive "sample-then-forget" rollout intervention designed to break this self-reinforcing exploitation loop and restore meaningful exploration.
More from Research
- '10% challenge' closes: match a model with 10% of its size and no weight training, $1k prize — tokenbender · 2026-11-13
- Photonics Atlas visualizes 4,537 optics milestones from 2,223 publications on an interactive timeline — jwt0625 · 2026-11-13
- GovAI paper examines what it would take for general-purpose AI to autonomously control military drones — sethlazar · 2026-11-13
- New paper argues AI agents must learn to hack to defend against cyber attacks — terryyuezhuo · 2026-11-13
- LeapQuant quantizes linear attention states to 8/4-bit training-free with 2x throughput — zealcaiden · 2026-11-13
- LeapQuant: 8/4-bit linear attention states, training-free, 2x throughput — SonglinYang4 · 2026-11-13