Why GRPO collapses on multi-turn agent training, and how to stabilize advantage estimation
burkov · x · 2026-10-20
Burkov highlights an in-depth article on a key fragility in agentic RL: GRPO, the most widely used method, was designed for single-turn tasks and tends to become unstable or collapse outright on multi-turn agent tasks requiring sequential decisions and environmental feedback. The article investigates more stable ways to estimate advantages — the signal telling the model which actions were better — and why current approaches fail in agentic settings. Essential reading for anyone building or depending on agent systems.
More from coding & agent
- Dev builds Overflow to turn leftover LLM subscription usage into open-source fixes — Nitjsefnie · 2026-10-20
- mlx-geospatial launches: fast geospatial Python library on Apple silicon via MLX — HankYeomans · 2026-10-20
- Mermaid's new Pele update brings major speedups and real-time color themes — draginol · 2026-10-20
- Vibe coding is everywhere, but new products aren't — discipline is the bottleneck — mariolefebvre · 2026-10-20
- RL fine-tuning Qwen3.5-0.8B for under $20 on HF Jobs lifts pass@1 from 10% to 36% — LysandreJik · 2026-10-20
- DeepSeek beats Claude and Codex on Google Maps scraping speed, at a steep price — teortaxesTex · 2026-10-20