Long-Horizon Agent Progress on Terminal-Bench
tetsuoai · x · 2026-07-14
The post compares the performance of long-horizon agents on Terminal-Bench:
- Earlier papers concluded there was vast room for improvement on this benchmark: the best of 15 models completed only 7 out of 46 tasks, averaging around 2.
- The author now claims Grok 4.5 has achieved 13 tasks and Fable 5 has reached 12, indicating significant progress in long-horizon agent execution over the past two months.
- Cost and execution metrics per task include roughly 9.9 million tokens, 231 episodes, and 85 minutes of wall-clock time.
- The author analyzes that Grok 4.5's "4.2x output token efficiency" actually underestimates the gains, as the primary cost on such benchmarks stems from input replay. Completing tasks in fewer steps reduces the accumulated context, substantially lowering overall input costs.
- The post speculates that Grok 4.5's improvements might stem from Cursor-related training data, specifically a massive amount of real-world agentic editing trajectories.
Related event: Grok 4.5 Tops Long-Horizon Terminal-Bench(3 posts)→
More from coding & agent
- Grok bot makes its first real purchase, spots hidden prompt injection at checkout and ignores it — msg · 2026-10-27
- Coding agents now make 60% of Vercel deployments, up from 3% in January — SuB8u · 2026-10-27
- Delta, Chase and Amazon are blocking AI agents — and the blockers may lose, predicts 6-stage war — alvelda · 2026-10-27
- Instinct launches Link balance to let agents spend within set budgets — jeff_weinstein · 2026-10-27
- Production lesson: a fluent AI translation can still be a broken document — WirelessLife · 2026-10-27
- One-prompt test: Grok Bot nails a premium DTC supplement landing hero — azed_ai · 2026-10-27