Terminal-Bench-Science lowers model pass rates by over 10%
sanmikoyejo · x · 2026-08-28
Terminal-Bench-Science discriminates between frontier models as effectively as Terminal-Bench 3.0 but pushes pass rates down by more than 10 points for every model. Specific drops: Opus 5 (43%→30%), GPT-5.6 Sol (34%→22%), and Fable 5 (34%→21%). This indicates the benchmark's tasks present a higher challenge for AI agents.
Related event: Stanford-led Terminal-Bench-Science Debuts; Claude Opus 5 Passes Just 30%(19 posts)→
More from Research
- OpenAI gives $5M to Simons Foundation's $15M push to help mathematicians navigate AI-era results — OpenAI · 2026-10-15
- DeepPhylo infers phylogenetic trees without unrealistic evolutionary assumptions — anshulkundaje · 2026-10-15
- Wander preprint explores graph foundation models via shared objective and random-walk interface — mmbronstein · 2026-10-15
- CHI Reviewer: AI Slop Is Real, but Unusually Rigorous Papers Still Getting Through — IanArawjo · 2026-10-15
- Into the Unknown: CMU benchmark finds 114 of 135 AI-written research papers fail integrity checks — arankomatsuzaki · 2026-10-15
- Benchling launches Science Foundries: multiplayer agent swarms for end-to-end science workflows — nlarusstone · 2026-10-15