HDM paper: "zero-or-T" routing cuts agentic token cost by 80% with no accuracy loss
pmttyji · reddit · 2026-11-16
A new paper introduces a zero-or-T thinking paradigm realized as a Hybrid Decision Model (HDM): instead of tuning how much a model reasons, a single frozen backbone either answers from implicit thinking in one forward pass (generating no tokens) or pays T tokens for full autoregressive reasoning when it's worth it.
How it works
- A lightweight self-routing block estimates the value of thinking — the expected utility gain of the slow mode over the fast one — from what the implicit thinking has already produced, running the slow mode only when that value exceeds a per-request price λ
- For episodic agentic tasks with no per-decision labels, the authors propose Calibrated Decision Policy Optimisation (CDPO), an RL recipe training the block from episode-level returns
Results
- Evaluated on MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77 and MiniWoB
- On JevBench, HDM-4B scores 0.857 vs Jev's 0.851 (+0.7%) at a median 0.179s vs 0.652s (72.5% faster)
- On the agentic MiniWoB benchmark, HDM-27B cuts token cost by 80.1%
Paper, code and HDM blocks are open-sourced on arXiv, GitHub and Hugging Face.
More from Research
- Dominik Grimm named TUM professor of ML for sustainable processes and materials — kmborgwardt · 2026-11-16
- Graph learning researcher Dexiong Chen joins HKUST Guangzhou as assistant professor — kmborgwardt · 2026-11-16
- Gamer: A Generalist VLA Foundation Model Trained on Tens of Thousands of Hours of Keyboard-Mouse Gameplay — pmttyji · 2026-11-16
- AI paper authors swarmed by predatory journal spam that smells like a pay-to-publish scam — letandrewcook · 2026-11-16
- Can we fix AI? Peter Bloem's new essay examines data, copyright and synthetic data paths — pbloemesquire · 2026-11-16
- Noumenal Labs open-sources hybrid-aif, a JAX active inference library for robotics — mjdramstead · 2026-11-16