Terminal-Bench 4.0 Released; Suspicions of Data Contamination
teortaxesTex · x · 2026-08-29
Terminal-Bench 4.0 has been released. A user points out that TB3 landed only 2 months ago, yet tasks were collected for over 4.5 months. TB4 contains no new tasks, only removals or modifications. This implies some tasks are 5+ months old and were published openly with solutions/graders on GitHub. While not confirming Zai trained on them, the user doubts that no companies have utilized these freely available, accessible environments (Harbour-compatible).
Related event: Terminal-Bench 4.0 released: Opus 5 leads, GLM 5.3 top open-source(6 posts)→
More from Research
- RL crucial for AGI; 10 Chinese firms capable of building it with GPUs — teortaxesTex · 2026-08-29
- MIT: AI agents differentiate and build persistent tech without communication — ProfBuehlerMIT · 2026-08-29
- Critique of AI Hype: Continual Learning data bottleneck and World Models necessity questioned — menhguin · 2026-08-29
- Karpathy's neural net training recipe still holds: errors can hide for a long time — iScienceLuvr · 2026-08-29
- TTPO: Test-Time Policy Optimization Boosts Qwen3-1.7B from 38.0% to 45.2% Without Labels — JFPuget · 2026-08-29
- Hugging Face adds Hindi, Indian English to ASR leaderboard based on spontaneous speech — aftahi_ai · 2026-08-29