AI Research Preference Models: Using Compute to Decide What Experiments to Run
anirudhg9119 · x · 2026-09-01
Paper "AI Research Preference Models" addresses the selection problem for AI research agents with limited compute budgets.
Key Details:
- Problem: Agents can propose more candidates than they can afford to evaluate. Progress depends on allocation strategy.
- Method: Introduces RPMs to predict promising candidates without running all.
- Inference-only: Reasons over plans, code, and history.
- Agentic: Runs small-scale pilot experiments.
- Results: Integrated into AIRA-dojo and evaluated on AIRS-Bench, scores increased from 0.684 to 0.711 and 0.729 respectively.
- Insight: In expensive-search domains, intelligence involves knowing what not to run. Selection quality scales with compute.
More from coding & agent
- Author Laurence Moroney: write your ugly first draft yourself, then let an LLM edit it — lmoroney · 2026-10-18
- Open-weight TYGER speech decision model makes 80ms turn-taking calls for full-duplex voice agents — solyarisoftware · 2026-10-18
- Devs: Opus 5.5 on medium beats Opus on max — eyishazyer · 2026-10-18
- He Stopped Telling Claude Code to Stop Making Up Timestamps—and Made It Impossible with a PreToolUse Hook — Grimmoner · 2026-10-18
- Cursor Super + Claude Pro + OpenCode: can this stack survive heavy autonomous coding? — DarioAmodeiWasTaken · 2026-10-18
- Coding AI wars: model is faster and smarter but 'dogshit at actual engineering' — MarcJSchmidt · 2026-10-18