TRACE: A New Reward Allocation Method for Long-Horizon Agents
SharonYixuanLi · x · 2026-07-19
TRACE is a turn-level reward allocation method designed for long-horizon agents, aiming to solve the credit assignment problem when relying solely on outcome-based rewards.
The core idea is to measure "how much more predictable the gold answer becomes given the current prefix" at each tool-call boundary using a frozen reference model. This change is converted into state values, and TD changes are used to assign positive/negative credit for each interactive step. It requires no critic training, process labels, Monte Carlo continuation, or heavy reliance on strong LLM judges.
The authors claim that in a pure RL setup without cold-start SFT, agentic mid-training, or live-web data, TRACE significantly boosts long-horizon search capabilities. On BrowseComp-Plus, Qwen3-4B improved from 7.2 to 35.6, and Qwen3-30B-A3B improved from 8.4 to 42.6, with the latter nearing the performance of Tongyi-DeepResearch-30B-A3B. When controlling for model, data, tools, and outcome reward, TRACE also outperforms GRPO, GSPO, and GiGPO.
Related event: TRACE: A New Credit Assignment Method for Long-Horizon Agents(3 posts)→
More from coding & agent
- Astra storyboards plus Minimax H3 per-shot generation boost video success rates — Hailuo_AI · 2026-09-11
- Codex tip: use Sol with Astra and Luna sub-agents to save usage — pvncher · 2026-09-11
- agents-best-practices: a provider-neutral Agent Skill for designing and auditing agentic harnesses — tom_doerr · 2026-09-11
- Cognition's SWE-2 uses a KKT duality argument in RL to shift the effort Pareto curve — YouJiacheng · 2026-09-11
- First-ever Three.js Conference lands in Paris, with a panel on AI-shortened design workflows — OdinLovis · 2026-09-11
- Agile co-author Ron Jeffries publishes 'Resist AI', urging developers to push back — mborch · 2026-09-11