Alibaba proposes I-SDPO to fix degenerate gradient problem in GRPO
heghbalz · x · 2026-08-17
Alibaba's latest paper introduces the I-SDPO framework to tackle the degenerate gradient problem in GRPO. When prompts are tough and every response fails, the policy gradient flatlines. While self-distillation helps, it can lock the model into a rigid style. I-SDPO makes distillation adaptive at the instance level: it uses privileged self-distillation only if an entire rollout group fails, flipping back to standard GRPO the moment a single attempt succeeds.
More from Research
- Five pharma giants federally train OpenFold3 on 20,000 private structures, +11 pts — jiqizhixin · 2026-10-16
- SWAN: a semantic-level text watermark that survives paraphrasing — burkov · 2026-10-16
- Interp researchers push back: 'networks are already clean' claim rests on dropout artifact — aryaman2020 · 2026-10-16
- LayerTrim: training-free pruning keeps reasoning LLMs intact on AIME — SeoulNatlUniv · 2026-10-16
- Simular presents neuro-symbolic computer use agents at BayLearn — xwang_lk · 2026-10-16
- Google's Hartmut Neven disputes chip test, outlines plan to link a brain to a quantum computer — Chris_Armstrong · 2026-10-16