Hybrid Decision Models: Only 30% of Queries Need Reasoning, Latency Cut 72.5%
hbouammar · x · 2026-11-15
A new paper from Huawei Noah's Ark Lab, University of Liverpool and UCL introduces Hybrid Decision Models (HDM): a frozen backbone that can answer directly with zero generated tokens, or switch to explicit reasoning only when the predicted benefit justifies the cost.
Key points:
- A lightweight self-routing block estimates the "value of thinking" — the expected utility gain from slow mode — trained via a new RL recipe called Calibrated Decision Policy Optimisation (CDPO) from episode-level returns.
- On JevBench, HDM-4B scores 85.7% accuracy vs Jev's 85.1%, with median latency of 179ms vs 652ms (72.5% lower), using explicit reasoning on only 30% of questions; on MiniWoB, HDM-27B cuts token cost by 80.1%.
- Unlike existing adaptive-reasoning methods that decide how much to think, HDM learns whether thinking is worth paying for — a distinction the authors argue matters greatly for agents making repeated decisions.
More from AGI Musings
- A barber only heard of ChatGPT last week: real-world AI adoption gap — AI_Andrew · 2026-11-15
- Researchers push back on Dwarkesh: defining AGI metrics is the hard part — dhadfieldmenell · 2026-11-15
- AI researcher: nobody can tell X replies are bot-written — and that's scary — iScienceLuvr · 2026-11-15
- Pedro Domingos mocks AGI hype: AI researchers never feel AGI is near — pmddomingos · 2026-11-15
- Pedro Domingos asks: how much false info is in your brain from trusting LLMs? — pmddomingos · 2026-11-15
- Blogger signs petition to reinstate Jensen Huang, citing his open-weight advocacy — esrtweet · 2026-11-15