Cambridge paper trains reasoning LLMs with information-gain rewards for cheaper experiment design
CambUni · hf · 2026-10-29
University of Cambridge paper 'Amortizing Sequential Experimental Design in Reasoning LLMs': LLMs struggle to adaptively gather information in experimental design tasks. The team amortizes sequential Bayesian experimental design (BED) into an LLM by training the reasoning that guides action selection with information-gain rewards, decoupling the generative model from the LLM's own beliefs so expected information gain can be optimized under any chosen model.
- Studies amortization design choices: generative-model constructions, reward estimation, hypothesis-space specification, multi-step objectives, and training/inference cost trade-offs.
- On Twenty Questions and related tasks, the resulting policies outperform inference- and training-time BED baselines at substantially lower compute and generalize to unseen categories.
- Demonstrates positive transfer to Medi-Q, a real-world medical diagnosis setting.
Related event: Cambridge Trains LLMs with Information-Gain Rewards for Experimental Design(2 posts)→
More from Research
- Stanford releases RT-J, a System 1 model predicting structured data in one forward pass — jure · 2026-10-30
- Signal65 Launches PINNACLE Coding Agent Benchmark: One Model Aces 97% of Samples, Fails All 30 Jobs — ryanshrout · 2026-10-30
- TEM: Transformer Energy-Based Models Claim to Outscale Diffusion Without Guidance — du_yilun · 2026-10-30
- VisualForce: robots infer grip force from soft gripper deformation, no force sensors needed — du_yilun · 2026-10-30
- Microsoft Research: coding agents' real bottleneck is understanding code, not writing it — Microsoft Research · 2026-10-30
- HALO (ICML 2026 Spotlight): certified-stable RL keeps humanoids coordinated with unpredictable humans — Scobleizer · 2026-10-30