Soft Data Duplication Blows Out Gradients: Root Cause of GRPO Training Collapse
ivan_bezdomny · x · 2026-08-02
The author diagnosed the root cause of the previous GRPO training collapse in Gemma. GRPO generates multiple outputs per input to compute gradients. When consecutive batches contain many of the same rare tokens (e.g., a specific bill name), it creates gradients in the exact same direction. This double-step backprop, amplified by momentum, blows out the model weights irrecoverably.
Related event: Developer Reveals LLM Fine-Tuning Fragility and Training Crashes(3 posts)→
More from Research
- Simplex Diffusion Models paper talk by EPFL/DeepMind's Justin Deschenaux on Oct 19 — ArnaudDoucet1 · 2026-10-16
- Meituan LongCat team's Data World synthesizes data environments to train data agents — meituan-longcat · 2026-10-16
- FDE-Gym benchmark shows huge gap on real AI delivery tasks: top model scores 0.563 — DeepLethe · 2026-10-16
- ForcingBooster halves computation of autoregressive video diffusion with patch-wise denoising — VAST-AI · 2026-10-16
- Baidu open-sources FlashMask: up to 1.55x faster than FlashAttention-4 for long-context training — BaiduResearch · 2026-10-16
- SemEval-2027 Task 9 opens: multimodal news framing analysis across 4 languages — preslav_nakov · 2026-10-16