Stanford-led Terminal-Bench-Science debuts: top model solves just 30% of 70 expert tasks
geoffwolfe · x · 2026-09-23
Researchers at Stanford and collaborators worldwide released Terminal-Bench-Science 0.1, extending the Terminal-Bench approach from software engineering to scientific research. The guiding principle: scientific-agent benchmarks should be built by the people who actually do the science, not model developers or data vendors.
- Task curation: only 70 tasks were accepted from 920 proposals, spanning life, physical, earth, mathematical, and engineering sciences — covering analyses, simulations, proofs, code, and data products drawn from real research workflows.
- Verifiability: every artifact is graded with reproducible, task-specific tests.
- Continuous benchmark: designed to evolve alongside frontier AI, creating a feedback loop between scientific needs and AI development.
- Results: the strongest evaluated system, Claude Opus 5, resolved just 30% of tasks, underscoring how hard real scientific workflows remain for current agents.
ReasonCore says it has Terminal-Bench-Science-style tasks in inventory for turning research workflows into hard, verifiable agent training and evaluation.
Related event: Stanford Releases Terminal-Bench-Science, Best Models Solve Only 30%(2 posts)→
More from coding & agent
- Loom founder launches Roma 2.0, a shared to-do list that orchestrates AI agents — thetripathi58 · 2026-11-12
- Sosumi Eval: Apple Docs Access Lifts Coding Agent Pass Rates from ~33% to 94-98% — JordanMorgan10 · 2026-11-12
- CypherGuard fixes structurally wrong LLM Cypher queries, hitting 88.5% on CypherBench — JeremyCMorgan · 2026-11-12
- Building an End-to-End Hedge Fund in Python with Claude Code: A 5-Step Workflow — mdancho84 · 2026-11-12
- Jeff Dean: comfortable with engineers spending up to $10M/year on AI tools — deedydas · 2026-11-12
- Developer builds VladOS, a personal life dashboard, by just describing it to an AI agent — haltakov · 2026-11-12