Tencent Hunyuan Releases Long-Horizon Agent Benchmark
Tencent-Hunyuan · hf · 2026-07-13
Tencent Hunyuan has released Long-Horizon-Terminal-Bench, a benchmark designed to evaluate agent capabilities in long-horizon terminal tasks.
Benchmark Design
- Features 46 long-horizon tasks across 9 scenario categories.
- Includes tasks like experiment replication, software engineering, multimodal analysis, interactive gaming, and scientific computing.
- Adopts a Terminal-Bench style but breaks tasks into finer sub-steps, providing dense intermediate rewards and partial credits.
Why It Matters
- Tasks typically require hundreds of episodes, lasting anywhere from tens of minutes to several hours.
- It heavily tests long-term planning, long-context management, and iterative debugging, rather than one-shot answering.
- On average, each task consumes 9.9 million tokens, roughly 231 episodes, and 85.3 minutes.
Results
- The strongest tested model achieved only 15.2% pass@1 under a partial reward threshold of 0.95.
- Under a full reward threshold of 1.0, the pass rate was 10.9%.
- Across all models, average pass rates were merely 4.3% and 1.7%, respectively.
The authors also analyzed failure modes and have made the benchmark openly available for future research.
Related event: Tencent Hunyuan Releases Long-Horizon Terminal Bench Exposing Model Limits(6 posts)→
More from coding & agent
- 'My intelligence stack': developer splits AI coding work across Opus 5.5, Fable and Astra with human review gates — sh_reya · 2026-10-27
- Baseten webinar: evaluate open models for coding agents by benchmarking on your own merged PRs — baseten · 2026-10-27
- Thorsten Ball on working with agents: run more evals, automate everything, use the best models — omarsar0 · 2026-10-27
- Where to draw the approval line for scheduled agent jobs — Reddit discussion — MemStrata · 2026-10-27
- 4 MCP servers worth a look: full-text YouTube search, Chrome session reuse, Framer editing — Prior_Engineering486 · 2026-10-27
- DeepMind paper: LLMs can't sample randomly on their own — pass random numbers from code instead — rohanpaul_ai · 2026-10-27