RL ablations: zero-padding fake negatives underperforms GRPO; good negatives need coherent failed trajectories
shangbinfeng · x · 2026-10-22
Ablations on RL with saturated data show non-zero advantage alone isn't enough: zero-padding with fake 0-reward samples inflates advantages but underperforms GRPO; swapping only the final answer hurts; higher temperature and rephrasing are inconsistent. A good negative needs a coherent, plausible failed trajectory.
More from Research
- Steering a single neuron in Olmo 3 bypasses post-trained assistant behavior — Sauers_ · 2026-10-22
- NVIDIA's Nemotron-Personas-Germany dataset trends on Hugging Face — nvidia · 2026-10-22
- CMU paper defines solver-sufficient symbols to cut hybrid robot planning cost — Yiyuan Pan · 2026-10-22
- NeurIPS Creative Track 2026 seeks sponsors to fill venue with AI art installations — dribnet · 2026-10-22
- JHU researchers preview their NLP papers coming to EMNLP 2026 in Budapest — mdredze · 2026-10-22
- Google DeepMind's Discovery team makes new progress on the elusive 6th Busy Beaver number — robertarail · 2026-10-22