Science's Last Exam: best of 8 frontier models scores 37.5/100, all score 0 on end-to-end research
anshulkundaje · x · 2026-11-06
The AItonomy Foundation, with researchers from 34 institutions (Harvard, MIT, Stanford and more), launched Science's Last Exam (SLE), an agent benchmark that opens fully on Nov 6.
- Unlike Q&A benchmarks, SLE gives agents real research goals plus actual tools (lab data, software) and grades them against hidden expert checks
- 113 public tasks across 9 disciplines and 6 capabilities (hypothesis discovery, protocol planning, scientific computing, software operation, wet-lab execution, observation), plus one end-to-end research task, written by PhD students and postdocs in 5 countries
- Reality check: the best of 8 frontier models scored just 37.5/100, and every model scored 0 on true end-to-end research
- Claude Opus 5.5 and GPT-6 Astra will be evaluated soon, pending bio-safeguard clearance
Related event: New Science Agent Benchmark SLE Leaves Frontier Models at Zero(2 posts)→
More from AGI Musings
- METR report finds AI is accelerating AI R&D at Anthropic; researcher now puts >50% odds on superintelligence within 18 months — EvanHub · 2026-11-06
- METR report on AI accelerating AI R&D flips a skeptic: >50% on superintelligence in 18 months — GaryMarcus · 2026-11-06
- Polymarket odds: only ~9% chance US enacts an AI safety bill by year-end — Polymarket · 2026-11-06
- Professor Matt Beane takes leave to build SkillBench for AI-era skill development — mattbeane · 2026-11-06
- Every publishes a new vision for human work after automation — every · 2026-11-06
- Alignment researchers debate threat models beyond scheming — MariusHobbhahn · 2026-11-06