From score forecasting to preference ranking: picking which of 15 ideas to run
anirudhg9119 · x · 2026-09-01
The motivation: generating a new ML idea or code change takes minutes, but testing it may require hours or days of GPU compute. If an agent proposes 15 ideas but can run only one, progress hinges on choosing the right one. Instead of predicting the exact score an experiment will achieve (a hard forecasting problem), the work reframes it as a preference problem: given several candidate solutions plus the research history, rank which candidate is most worth executing next.
More from Research
- Terence Tao on 'War Footing' as OpenAI Claims Navier-Stokes Solve Shakes Mathematics — AlexTensor · 2026-10-18
- Univ. of Tokyo paper: why your recorded voice feels eerie — a vocal uncanny valley — ssh4net · 2026-10-18
- MIT's SANDO Drone Planner Avoids Every Moving Obstacle Across 12 Real Flights — Olivier__OG · 2026-10-18
- No-backprop pretraining speedup of 2.3x claimed, inspired by mech interp essay — CatAstro_Piyush · 2026-10-18
- Models Can Be Trained to Sense Activation Probes and Generalize Across GPUs Zero-Shot — Sauers_ · 2026-10-18
- Simulating a Turing machine with addition and multiplication by encoding the tape as a number — jessi_cata · 2026-10-18