Meta's MLE agents ran 1,097 experiments on production rankers with 29.6% acceptance rate
dair_ai · x · 2026-11-19
Meta's paper on autonomous ML engineering agents tested on production search rankers: agents propose, implement, train and evaluate model changes with engineers controlling objectives and final review. One campaign ran 1,097 experiments and accepted 325 (29.6% vs 15.6% earlier); the best model lifted search volume 0.23% in live A/B tests. Key lessons: parallel training runs, objectives calibrated to online impact, and careful context engineering — harness design matters as much as the underlying model.
More from coding & agent
- Kong launches Volcano, a 'self-healing' cloud for agents after handling 500T AI tokens a year — EXM7777 · 2026-11-19
- Grok bot trained on Dan Koe's system turns one idea into a week of content — VibeMarketer_ · 2026-11-19
- Matt Pocock unveils VECTOR method for adopting AI on any task, seeks early feedback — mattpocockuk · 2026-11-19
- 11 system security concepts every engineer should know, from OAuth to Zero Trust — goyalshaliniuk · 2026-11-19
- Open-source Nango connects AI agents to 1,200+ APIs with one auth flow — aftahi_ai · 2026-11-19
- Joseph Jacks: If You're Not Shipping Like a 50-Person Team, You're Using AI Wrong — JosephJacks_ · 2026-11-19