GRPO post-training collapses model distributions, clashing with long-horizon agents
zsakib_ · x · 2026-10-16
yacinelearning explains that post-training with GRPO collapses the model onto a tight set of potential answers — a "spooky sharpening" of the distribution — which is sometimes exactly what you want. But in 2026, models increasingly run in long-running harnesses requiring LLMs to consider diverse solutions and act compositionally, putting this collapse at odds with agentic usage.
More from coding & agent
- Inflight launches: a feedback layer that turns prototype comments into Claude tasks — soleio · 2026-10-16
- Apple code signing drags for hours; official advice: just wait a week — pjausovec · 2026-10-16
- Hermes: a 4B model with 8K context got worse with more test-time compute, pointing to contextual reasoning — mangahomanga · 2026-10-16
- Flux 3 Klein's Default Templates Are Broken Without Its Prompt Upscaler — StrictMove5839 · 2026-10-16
- 21-person team ships hundreds of PRs weekly with AI writing 97% of the code — rseroter · 2026-10-16
- ClosedMCP: Set Up MCP Servers Once, Share Them Across Your Whole Team — Ok_Sprinkles1301 · 2026-10-16