ProAS Fixes Uniform Credit Assignment in Critic-Free Agentic RL
LINs-lab · hf · 2026-10-27
Researchers from Zhejiang University, Westlake University, and Huawei Noah's Ark Lab propose ProAS (Progress-Aware Advantage Shaping), a plug-and-play framework for multi-turn agentic reinforcement learning.
- Problem: critic-free algorithms like GRPO suffer from a uniform credit assignment dilemma — trajectory-level advantages are broadcast equally to all actions, reinforcing structurally unreasonable actions in preferred trajectories and ignoring each action's downstream deviations in unprefixed ones.
- Method: ProAS uses a dynamic reference graph to mask unreasonable incentives and redistribute penalties asymmetrically, without critic overhead.
- Results: reported gains in success rate and convergence speed on multi-turn agent tasks.
More from coding & agent
- Developer banned after using Codex on his own logs; mark_k asks OpenAI to investigate — mark_k · 2026-10-27
- rauchg on why AI products must hide the org chart, like Facebook fought Conway's Law — soleio · 2026-10-27
- Agent 'Astra' caught forking preact and opening an unauthorized GitHub PR — basedjensen · 2026-10-27
- Blogger lets Grok Bot make all his calls for a day: $35/month cut and gym cancellation — Scobleizer · 2026-10-27
- Void v1.3 ships JSX routes, email aliases, and a wave of Cloudflare fixes — cnakazawa · 2026-10-27
- Dev says building game worlds on Spawn with Opus enables endless shooter maps — TAbrodi · 2026-10-27