DeltaTTG Uses a Token-to-Go Progress Meter to Beat GRPO by 7.2 Points on Multi-Turn Agents
RUC-AIBOX · hf · 2026-10-14
NTU researchers propose DeltaTTG to tackle sparse terminal rewards in multi-turn agent RLVR. The method trains a shared frozen progress meter whose supervision target is Token-to-Go (TTG)—the number of tokens remaining until task completion—as an observable proxy for remaining effort, using low-cost supervision from recorded trajectories of diverse policies without task-specific rules. During actor training, changes in the meter's estimates redistribute outcome-derived advantages across steps without altering rewards, preserving advantage signs and token-average scale.
On ALFWorld, PaperSearchQA, and BFCLv3, DeltaTTG beats GRPO by 5.7 and 7.2 average points with Qwen3-4B and Llama-3.1-8B-Instruct, outperforms all compared RL baselines, and keeps GPU-hours close to GRPO. It also improves sample efficiency and out-of-domain performance, and the meter distinguishes step quality and guides test-time action selection. Code is available.
Related event: DeltaTTG: A Token-Based Progress Meter Improves Multi-Turn Agent RL(2 posts)→
More from coding & agent
- DeepStation meetup demos OpenAI's post-DevDay Codex tools live for developers — MicahBerkley · 2026-10-23
- Open-source semantic search makes GitHub issue search 2x faster, 8x more accurate — jamesmurdza · 2026-10-23
- HarnessRouter open-sources a unified API to run Codex, Claude Code and more as agent backends — tom_doerr · 2026-10-23
- React Scan author teases new tool for long-standing React perf pain — aidenybai · 2026-10-23
- Coinbase launches agent page letting ChatGPT, Claude and Grok trade crypto and stocks for you — MurrLincoln · 2026-10-23
- Malware found in GitHub DaisyUI-MCP repo: luajit.exe trojan steals credentials — AveryFreeman · 2026-10-23