Terminal-Bench 4.0 Released; Suspicions of Data Contamination
teortaxesTex · x · 2026-08-29
Terminal-Bench 4.0 has been released. A user points out that TB3 landed only 2 months ago, yet tasks were collected for over 4.5 months. TB4 contains no new tasks, only removals or modifications. This implies some tasks are 5+ months old and were published openly with solutions/graders on GitHub. While not confirming Zai trained on them, the user doubts that no companies have utilized these freely available, accessible environments (Harbour-compatible).
Related event: Terminal-Bench 4.0 Released: Opus 5 Leads, GLM 5.3 Top Open-Source Model(14 posts)→
More from Research
- U of T team mines 122,194 patient reviews across 45 Ontario hospitals with NLP to expose satisfaction and equity patterns — ishtiaqueSIA · 2026-10-16
- University of Toronto hiring AI postdoc: builders of interventions, not just benchmarkers, due Oct 30 — ishtiaqueSIA · 2026-10-16
- WorldMemArena Accepted at NeurIPS: Memory Storage ≠ Memory Use — xwang_lk · 2026-10-16
- Mech Interp Veteran Rethinks Alignment, Misgeneralization, CoT Oversight — akbirthko · 2026-10-16
- WEDGE paper generates training environments conditioned on navigation agent failures — _akhaliq · 2026-10-16
- RExBench: best coding agent completes only 33% of autonomous research extensions — najoungkim · 2026-10-16