AutoResearchExam benchmark tests 24-hour autonomous AI research

Bespoke Labs and Alex Dimakis' team released AutoResearchExam, a benchmark of 29 open-ended ML research tasks where agents get 24 hours of machine access, evaluated via a hidden test set and a new AUARC metric to measure autonomous research capability.

2026-09-10 ~ 2026-09-10 · 4 related posts

• AutoResearchExam: a 24-hour benchmark finds AI research agents overfit, with Fable 5.1 edging Astra — AlexGDimakis · 2026-09-10
• AutoResearchExam: a 24-hour benchmark reveals how AI agents really do research — AlexGDimakis · 2026-09-10
• Bespoke Labs launches AutoResearchExam: 29 tasks to benchmark 24-hour autonomous ML research agents — AashaySachdeva · 2026-09-10
• AutoResearchExam uses hidden test sets to study how AI agents do 24-hour research — AlexGDimakis · 2026-09-10