TRACE: A New Reward Allocation Method for Long-Horizon Agents
SharonYixuanLi · x · 2026-07-19
TRACE is a turn-level reward allocation method designed for long-horizon agents, aiming to solve the credit assignment problem when relying solely on outcome-based rewards.
The core idea is to measure "how much more predictable the gold answer becomes given the current prefix" at each tool-call boundary using a frozen reference model. This change is converted into state values, and TD changes are used to assign positive/negative credit for each interactive step. It requires no critic training, process labels, Monte Carlo continuation, or heavy reliance on strong LLM judges.
The authors claim that in a pure RL setup without cold-start SFT, agentic mid-training, or live-web data, TRACE significantly boosts long-horizon search capabilities. On BrowseComp-Plus, Qwen3-4B improved from 7.2 to 35.6, and Qwen3-30B-A3B improved from 8.4 to 42.6, with the latter nearing the performance of Tongyi-DeepResearch-30B-A3B. When controlling for model, data, tools, and outcome reward, TRACE also outperforms GRPO, GSPO, and GiGPO.
Related event: TRACE: A New Credit Assignment Method for Long-Horizon Agents(3 posts)→
More from coding & agent
- Practical tip: use max effort for planning, xhigh for implementation in Claude Code — majidmanzarpour · 2026-10-27
- Chrome team explains WebMCP: agents get tools from your website, not a replacement for MCP — rseroter · 2026-10-27
- Agent-Native apps: why agents bypassing your UI can break business rules — rseroter · 2026-10-27
- 'My intelligence stack': developer splits AI coding work across Opus 5.5, Fable and Astra with human review gates — sh_reya · 2026-10-27
- Baseten webinar: evaluate open models for coding agents by benchmarking on your own merged PRs — baseten · 2026-10-27
- Thorsten Ball on working with agents: run more evals, automate everything, use the best models — omarsar0 · 2026-10-27