Credit assignment, not better data or entropy, looks like the bottleneck for agentic RL

burny_tech · x · 2026-07-28

The post points to a long piece arguing that credit assignment is the key bottleneck when moving from reasoning models to agentic tasks.
In pure reasoning settings, models can still improve with larger output length, bigger group size, longer training, and lower-entropy/different-difficulty data.
But for agentic tasks, the author says the usual knobs — changing data, adding KL to reduce policy drift, increasing entropy for exploration, or tweaking the recipe — often fail to solve the problem.
The analysis suggests that simple single-turn evaluation may miss the issue, and that progress likely requires new data design, better rubric/LLM-judge setup, and more careful rollout sampling.
The post also notes a practical tradeoff: longer training horizons such as 128k–256k make high-quality rubrics and judge calls expensive, slowing both training and validation.

More from coding & agent