Google's ScientistTwo solves 80.4% of 107 top-venue ML problems autonomously
thisdudelikesAI · x · 2026-09-22
Google unveiled ScientistTwo, an autonomous scientist pipeline: given a research problem, it surveys the state of the art, identifies limitations, generates hypotheses, writes code, runs experiments across datasets, performs ablations, drafts the paper, and iterates through a simulated peer-review loop where a rebuttal agent runs new experiments to answer reviewer criticisms. A meta-reviewer gates each round with no human in the loop after the problem statement.
Tested on 107 problems drawn from accepted ICLR, ICML, and NeurIPS papers:
- Advanced 86 of 107 (80.4% success rate)
- 25.2% average relative gain over human state-of-the-art
- Papers score above the average accepted ICLR 2026 and NeurIPS 2025 paper under the Stanford evaluation
Related event: Google's ScientistTwo Automates Research at 80% Success Rate(2 posts)→
More from AGI Musings
- Prediction: hospitals will keep their own tested open-model formularies in two years — MaziyarPanahi · 2026-11-12
- Packy revives Qwest CEO's 1998 'no bandwidth glut' quote for the AI oversupply debate — prestonpesek · 2026-11-12
- Stanford's James Zou joins The Economist podcast on AI agents in science — anshulkundaje · 2026-11-12
- a16z maps the B2A economy: AI agents are now most of web traffic and soon its buyers — venturetwins · 2026-11-12
- Jason: Anthropic is rebuilding every SaaS product into Claude—free for life — smtabatabaie · 2026-11-12
- Fleuret: if AI designs drugs as fast as it makes dumb videos, regulation will buckle — francoisfleuret · 2026-11-12