Terminal-Bench 4.0 Released: Opus 5 Leads, GLM 5.3 Top Open-Source Model

Terminal-Bench 4.0 was officially released on Aug 29, covering 66 tasks across science, software, ML, ops, security, hardware and media. With higher difficulty, scores dropped broadly: Claude Opus 5 ranks first at 51.8% accuracy (versus near 90% on the old version), beating Fable 5; GLM 5.3 matches Fable 5 within error bars, surpasses GPT-5.6, and is rated the best open-source model, with @heypearlai noting its efficiency rivals Opus 5 despite being free and open-source. @DimitrisPapail calls the benchmark the ImageNet of the agent domain.

Confirmed

• Terminal-Bench 4.0 released with 66 tasks spanning science, software, ML, ops, security, hardware and media
• Per @eyishazyer, scores fell broadly: Claude Opus 5 leads at 51.8%, just over half the tasks
• Opus 5 beats Fable 5; GLM 5.3 ties Fable 5 within error bars, exceeds GPT-5.6 and is rated the best open-source model; @heypearlai notes its free/open-source efficiency rivals Opus 5
• Updates include: calibrating per-task compute resources (time, CPU, memory) with a unified 8-hour timeout; fixing 19 broken or unreliable tasks; removing 8 saturated or problematic tasks due to saturation from 100% accuracy, high refusal rates, public solutions and other compatibility issues
• The new version markedly reduces agent timeouts and errors, lowering measurement noise
• All analysis in the official blog post is based on Terminal-Bench 4.0 data hosted on Harbor Hub
• Per @cwolferesearch and @zainhas, TB4 shipped only about a month after the previous version and the benchmark is shifting to continuous-style updates; TB4 also debuts a new "waffle chart" visualization showing how models perform across tasks over multiple attempts, making it easy to see which tasks are saturated and which remain at the frontier

Unconfirmed

• @teortaxesTex noted that TB3 landed only two months ago but its tasks were collected over 4.5+ months; TB4 adds no new tasks, only removing or modifying some, meaning some tasks are 5+ months old and public on GitHub with solutions/scorers—raising possible contamination concerns, though this remains a user质疑 without official response or evidence

Why it matters

• Rapid iteration counters benchmark saturation, keeping evaluation pace aligned with model progress, a direction also emphasized by @yacineMTB
• @SorosAhaverom praised the strategy and explored low-cost, low-compute alternative evaluation options for individual developers and small teams
• Visualizations like the waffle chart offer intuitive references for model selection and evaluation
• GLM 5.3's standing as the best open-source model is a useful signal for the open-source ecosystem

2026-08-29 ~ 2026-08-30 · 14 related posts

• Terminal-Bench 4.0 Released, Benchmark Iteration Catches Up with Model Development — yacineMTB · 2026-08-29
• [source] Terminal-Bench 4.0 Released: Opus 5 Leads, GLM 5.3 Best Open Source — himanshustwts · 2026-08-29
• Terminal-Bench 4.0: Calibrating Resources, Fixing Tasks, Removing Saturated Items — himanshustwts · 2026-08-29
• Terminal-Bench 4.0 released: calibrates resources, fixes tasks, removes saturation — giansegato · 2026-08-29
• Terminal Bench 4.0 Released: GLM-5.5 Matches Fable 5 Performance — SorosAhaverom · 2026-08-29
• Terminal-Bench 4.0 Released; Suspicions of Data Contamination — teortaxesTex · 2026-08-29
• [source] Terminal-Bench 4.0: GLM-5.3 Surpasses GPT-5.6 in New Ranking — eyishazyer · 2026-08-29
• Terminal-Bench 4 released: bug fixes and calibrations for agentic benchmarks — DimitrisPapail · 2026-08-30
• Terminal-Bench 4 introduces waffle view for model task saturation — zainhas · 2026-08-30
• Terminal Bench 4 Released: Removes 8 Tasks for Saturation and Refusals — cwolferesearch · 2026-08-30
• [source] Terminal-Bench 4.0 Out: Calibrated Resources, Saturated Tasks Removed, Continuous Releases — cwolferesearch · 2026-08-30

3 near-duplicate retellings: heypearlai · heypearlai · sanmikoyejo