Let the policy write wrong solutions: negative rollouts beat GRPO by 9.0% on Qwen3-1.7B using only saturated MATH data
shangbinfeng · x · 2026-10-22
Doing RL only on saturated MATH problems (8/8 rollouts correct), asking the policy itself to write a wrong solution and adding it to the group as a negative works best: +9.0% over GRPO on Qwen3-1.7B and +6.4% on 4B, averaged over 8 math, reasoning and IF benchmarks. Negative rollouts beat zero-padding, higher temperature, reward models and irrelevant-info baselines.
More from Research
- Ask-to-Distill: synthesizing questions for answer-free on-policy distillation — UIUC-CS · 2026-10-22
- RFD4 team swaps self-conditioning for recurrent updates, echoing looped flows — AllThingsApx · 2026-10-22
- MIT ML technique breaks records for 2-6 week weather forecasts by learning to correct model errors — LesterMackey · 2026-10-22
- Harvard undergrad's ML subseasonal weather forecasting work, started at Microsoft Research, lands in Nature — LesterMackey · 2026-10-22
- Chatlet: generative UI that emerges from group chats to power collaborative decisions — HaijunXia · 2026-10-22
- If your research idea is concrete enough, agents can mostly execute it, says AI practitioner — burny_tech · 2026-10-22