TRACE: A New Reward Allocation Method for Long-Horizon Agents
SharonYixuanLi · x · 2026-07-19
TRACE is a turn-level reward allocation method designed for **long-horizon agents**, aiming to solve the **credit assignment problem** when relying solely on outcome-based rewards. The core idea is to measure "how much more predictable the gold answer becomes given the current prefix" at each tool-call boundary using a **frozen reference model**. This change is converted into state values, and TD changes are used to assign positive/negative credit for each interactive step. It requires no critic training, process labels, Monte Carlo continuation, or heavy reliance on strong LLM judges. The authors claim that in a pure RL setup without cold-start SFT, agentic mid-training, or live-web data, TRACE significantly boosts long-horizon search capabilities. On BrowseComp-Plus, Qwen3-4B improved from 7.2 to 35.6, and Qwen3-30B-A3B improved from 8.4 to 42.6, with the latter nearing the performance of Tongyi-DeepResearch-30B-A3B. When controlling for model, data, tools, and outcome reward, TRACE also outperforms GRPO, GSPO, and GiGPO.
Related event: TRACE: New Credit Assignment for Long-Horizon Agents(2 posts)→
More from coding & agent
- ZooData’s API takes a URL and returns structured data in the same request — dr_cintas · 2026-07-21
- ZooData turns any web page into structured JSON for agents with one API call — dr_cintas · 2026-07-21
- Side-by-side model test lands both answers on the first try, then shifts to loop engineering — glenbeer · 2026-07-21
- TokenPrint turns Qwen inference into a DevTools-style visual debugger — Rich-Fruit-326 · 2026-07-21
- This week's must-read AI papers span agents, long-context RL, and robot policies — TheTuringPost · 2026-07-21
- Mingle uses MCP to let agents find collaborators without profiles — PassionGlittering106 · 2026-07-21