TurnSight: Turn-Level Hindsight Self-Distillation for Tool-Integrated Reasoning
RUC · hf · 2026-08-05
Tool-Integrated Reasoning (TIR) enables LLMs to solve complex tasks via iterative tool interactions. However, existing RL methods often rely on trajectory-level supervision, limiting fine-grained credit assignment in long-horizon TIR scenarios.
Current self-distillation approaches typically derive context from ground-truth answers or retrieved skills, which may not reflect the agent's actual visited states. Moreover, token-level supervision fails to capture the turn-level structure of tool interactions. To address this, researchers proposed TurnSight, a turn-level hindsight self-distillation framework.
TurnSight derives supervision directly from execution-conditioned hindsight. It constructs multiple hindsight views with different lookahead horizons and selects reliable supervision through cross-horizon directional agreement. The selected signal is then normalized across sibling rollouts and used to adaptively modulate RL advantages while preserving their original optimization direction. Experiments on three benchmarks demonstrate its effectiveness.
More from coding & agent
- LangChain Releases Open SWE: An Open-Source Enterprise Coding Agent Framework — hwchase17 · 2026-08-05
- GitHub Copilot CLI Update: Fixes MCP Startup Stalls and UI Tweaks — copilot-cli-release-app[bot] · 2026-08-05
- Surviving the First Infrastructure Incident Caused by a Coding Agent — yassineyousfi_ · 2026-08-05
- Testing Bolt's Template Market: How Do AI-Generated CRM Apps Hold Up? — HeyAmit_ · 2026-08-05
- AI Agent Autonomously Reproduces and Improves Paper in 5 Days, 1100+ Actions — CodeByPoonam · 2026-08-05
- PCSD: Persistent Consistency Self-Distillation Tackles Sparse Rewards in Agentic RL — BIT · 2026-08-05