NVIDIA's HDL Cuts RLVR Token Cost 2.5x by Localizing Where Models Change Their Mind
nvidia · hf · 2026-09-30
NVIDIA proposes Hindsight-Divergence Localization (HDL) for RL with verifiable rewards: hindsight-induced shifts in token log-likelihoods identify branch points where the policy reconsiders earlier choices, and training groups are filled with suffix-only continuations from those positions. Across three models on math, code, and agent tasks, HDL reduces generated tokens up to 2.5x and speeds rollout 1.8x versus GRPO, while improving performance—up to 12.5 points on agent tasks.
More from Research
- Open-source TotalSegmentator spots a 42mm dilated aorta on a routine chest CT — MaziyarPanahi · 2026-11-16
- Tivadar Danka Publishes 4,000-Word Visual Guide to Graph Theory for Machine Learning — TivadarDanka · 2026-11-16
- METR time-horizon chart updated: doubling every 1.7 months since Opus 4.5, 'Model 4' ~38h — ben_j_todd · 2026-11-16
- GTAgentClinic: one-line install, plug in Claude Code or Codex for a 3D hospital shift — Michael_D_Moor · 2026-11-16
- 28-model tracking table: half publish no training cutoff, Gemini 4 Carbon 8 months stale at launch — Joozio · 2026-11-16
- FutureSim Q3: LLMs now genuinely improve from experience, Opus 5.5 tops at 50% — rickasaurus · 2026-11-16