Meta's RPM paper: treat experiments as tree nodes and teach agents "research taste"
_lewtun · x · 2026-09-04
Hugging Face researcher Lewis Tunstall highlights Meta's new paper on Research Preference Models (RPMs), which tackles a key question for automated R&D: how to instill "research taste" in agents. Existing evidence shows agents get stuck in local optima — scaling Nano-GPT speed-runs showed agents burning compute on hyperparameter tuning instead of novel ideas, and PostTrainBench found most models favor algorithmic tweaks over curating high-quality datasets. The RPM approach treats each experiment as a node in a tree, scores nodes via downstream evals, mutates nodes to generate candidates, and uses an RPM (an LLM) to judge which experiments are worth pursuing.
More from AGI Musings
- Jensen Huang to German Chancellor: you can't regulate technology that doesn't exist — robleclerc · 2026-10-21
- Richard Socher: AI could compress a century of scientific breakthroughs into a decade — RichardSocher · 2026-10-21
- Ethan Mollick: AI's work capacity doubled 8 times in 30 months, METR benchmark maxed out — One Useful Thing (Ethan Mollick) · 2026-10-20
- Time for Guardrails on Government Too: Rethinking Who AI Policy Constrains — neil_chilson · 2026-10-20
- Steam needs real quality control on AI slop, not an AI art ban, argues designer — AIandDesign · 2026-10-20
- Why personal AI agents fall into a retention dead zone between dopamine and infrastructure — signulll · 2026-10-20