UT Austin proposes Horizon-Scalable Q-Learning for long-horizon offline RL
UTEXAS · hf · 2026-11-10
UT Austin researchers (Haoran Xu, Amy Zhang, et al.) propose Horizon-Scalable Q-Learning (HQL). Learning accurate off-policy Q-functions from offline data gets harder as horizons grow: one-step TD accumulates bootstrapping errors, while multi-step TD adds bias from suboptimal behavior policies with no fixed step count fitting all trajectories. HQL extends a target through a dataset action only when the critic values it at least as highly as the best-of-N actions from a behavior-cloning policy, automatically balancing one-step and multi-step returns for better long-horizon value learning.
More from Research
- Preprint: Learning Tool-Output Projections from Agent Traces Cuts Billed Tokens up to 56% — solyarisoftware · 2026-11-10
- Shanghai AI Lab's Sage-Taiyi tops AI co-scientist benchmark, lifts reaction yields to 84% — Shanghai-AI-Laboratory · 2026-11-10
- SIRIS benchmark: LLMs can't reliably tell when simulator errors invalidate design recommendations — MicrosoftResearch · 2026-11-10
- How many exposures does a world model need? WARC learns unseen game mechanics in ~80 exposures — third-intelligence · 2026-11-10
- Tencent Hunyuan's FewStepRL adapts RL fine-tuning to few-step diffusion models — Tencent-Hunyuan · 2026-11-10
- BioEmu2 simulations show ligand-triggered conformational shifts in maltose binding protein — FrankNoeBerlin · 2026-11-10