Training Crash: GRPO Reward Drops to Zero, Devolving Model Outputs into Gibberish
ivan_bezdomny · x · 2026-08-02
A developer shared a classic training collapse issue encountered while fine-tuning a model (e.g., Gemma) using reinforcement learning: the GRPO reward suddenly drops to zero even when the process previously looked stable.
Accompanying this metric crash, the model completely loses its ability to generate coherent text, with all outputs devolving into repetitive gibberish. The author notes this is a clear sign that the model has entered a state from which it cannot recover.
Related event: Gemma Fine-tuning Fails as GRPO Reward Drops to Zero(2 posts)→
More from Research
- Elegant Sliding Window Aggregation Algorithm for Non-Invertible Operators Goes Viral — Paimaamu · 2026-10-16
- A $50 site of 50,000 fake stats got cited by ChatGPT for 7 months — metehan777 · 2026-10-16
- DeepMind wires Antigravity into AlphaFold to speed up science, finds novel antimicrobial peptide — pushmeet · 2026-10-16
- It's not gradient descent: burny_tech corrects the diagnosis to RLHF mode collapse — burny_tech · 2026-10-16
- A beginner-friendly introduction to causal inference kicks off, Part 1 — Nasereliver · 2026-10-16
- FBI method trains for Pass@K with one sample per update at 12.3% of the cost — EmtiyazKhan · 2026-10-16