Meta's MLE agents ran 1,097 experiments on production rankers with 29.6% acceptance rate

dair_ai · x · 2026-11-19

Meta's paper on autonomous ML engineering agents tested on production search rankers: agents propose, implement, train and evaluate model changes with engineers controlling objectives and final review. One campaign ran 1,097 experiments and accepted 325 (29.6% vs 15.6% earlier); the best model lifted search volume 0.23% in live A/B tests. Key lessons: parallel training runs, objectives calibrated to online impact, and careful context engineering — harness design matters as much as the underlying model.

Related event: Meta's MLE agents ran 1,097 experiments on production rankers with 29.6% acceptance rate(2 posts)→

Original post →

More from coding & agent

coding & agent channel →