Terminal-Bench-Science: GPT-6 Astra 63%, most open-weights models under 10%
ArtificialAnlys · x · 2026-10-28
Artificial Analysis published Terminal-Bench-Science scores: the benchmark tests whether agents can execute scientific research workflows in a terminal, with expert-curated tasks that pass only if every test passes.
GPT-6 Astra (max) leads at 63%, followed by Claude Opus 5.5 (max) at 59% and GPT-6.1 Sol (max) at 58%. Scores fall off a cliff below: Kimi K3.1 Pro (weights unreleased) at 34%, Claude Haiku 5.5 (max) at 20%; the best released-weights model, DeepSeek V4.1 Flash 1017 (max), scores just 13%, and most open-weights models score under 10% — the open/closed gap in agentic science is far wider than on composite indexes.
More from Models
- Debate: Claude isn't 'lobotomized' — inference stack updates may shift old-model outputs — JasonBotterill · 2026-10-28
- AI labs near IPO, yet LLMs still produce 'astonishingly terrible prose' by default — GabGarrett · 2026-10-28
- Rebuttal: Kimi 3.1 Nears Fable on Frontier Code 2, Contradicting 'China Models Crushed' Claim — bookwormengr · 2026-10-28
- Open model Flint hits #2 trending on Hugging Face, team outlines next steps — josh_tobin_ · 2026-10-28
- Artificial Analysis Intelligence Index v5 widens open-weight gap from 3 to 12 points — ResearchCrafty1804 · 2026-10-28
- YacineMTB rails: even the best LLMs are 'complete dogshit' and he's tired of it — yacineMTB · 2026-10-28