Terminal-Bench 4.0 Unifies 8-Hour Timeout to Curb Benchmark Gaming

Terminal-Bench 4.0 unifies an 8-hour agent timeout for all tasks after Vals revealed that most score discrepancies came from labs loosening timeout limits to inflate results, while standard-timeout evaluators got lower scores.

2026-09-17 ~ 2026-09-17 · 2 related posts

• Terminal-Bench 4.0 Sets Flat 8-Hour Agent Timeout After Lab Timeout Discrepancies Skewed Scores — langstonnashold · 2026-09-17
• Evals Shouldn't Reward Better Infra: 90% of Terminal-Bench Mismatches Came From Longer Lab Timeouts — xeophon · 2026-09-17