TreeOPSD: Tencent and Tsinghua's critic-free RL for long-horizon LLM agents
Tencent-Hunyuan · hf · 2026-10-19
TreeOPSD, from Tencent and Tsinghua, enables long-horizon agentic RL without critics. It treats each interaction turn as a tree node, branches alternative continuations from shared prefixes, and recursively backs up terminal rewards into implicit turn-level credit, mapped to a closed-form bounded teacher distribution via likelihood ratios. Minimizing teacher-student divergence bounds sequential updates, addressing sparse credit assignment and policy staleness from terminal-only rewards.
More from coding & agent
- Voice booking agent on ElevenLabs + AWS: tool-turn latency hits 5s P95, how to fix? — matias_flyfishing · 2026-10-19
- Every engineering leader is building the same AI automations right now — generativist · 2026-10-19
- 'Prayer-based programming' emerges as AI coding's latest self-deprecating meme — lxfater · 2026-10-19
- Developer: I keep playing 'human API' in my AI workflows — jacalulu · 2026-10-19
- Vibe coding a parent's TV: Claude Code strips 25 Xiaomi packages, kills boot ads — HarperSCarroll · 2026-10-19
- Sentry co-founder's AI writing workflow: dictate ideas, let an agent outline, write it yourself — lmoroney · 2026-10-19