Survive or Collapse: data gating, not rewards, drives self-play RL collapse
shangbinfeng · x · 2026-10-17
The paper Survive or Collapse, accepted to NeurIPS 2026, studies why self-play RL for LLMs keeps collapsing.
- Most fixes target the reward signal, but the authors argue that's the wrong lever; the binding constraint is data gating — deciding which self-generated tasks enter the training pool.
- Collapse traces back to a feedback loop of self-generated errors that compound once bad tasks pollute training.
- Takeaway: stabilize self-play by controlling training-pool admission, not just reward shaping.
More from Research
- APS Global Physics Summit opens Neural Systems session, abstracts due Oct 2026 — chklovskii · 2026-10-17
- RAX open-sourced: frozen coding agents learn robot tasks via online exploration, zero demos — du_yilun · 2026-10-17
- Open textbook "Elements of Robotics" by Ben-Ari & Mondada now freely available — burkov · 2026-10-17
- U. Tokyo's TwinX robot: one wire links two wheeled-legged bots for morphing and teamwork — LexiLove · 2026-10-17
- NVIDIA open-sources midtraining recipe: 500B tokens lift Nemotron agentic benchmark scores — denisyarats · 2026-10-17
- PySR v2 ships after two years of work: plugins, custom types, much faster — MilesCranmer · 2026-10-17