Tencent Hunyuan Releases Long-Horizon Agent Benchmark

Tencent-Hunyuan · hf · 2026-07-13

Tencent Hunyuan has released Long-Horizon-Terminal-Bench, a benchmark designed to evaluate agent capabilities in long-horizon terminal tasks.

Benchmark Design

Features 46 long-horizon tasks across 9 scenario categories.
Includes tasks like experiment replication, software engineering, multimodal analysis, interactive gaming, and scientific computing.
Adopts a Terminal-Bench style but breaks tasks into finer sub-steps, providing dense intermediate rewards and partial credits.

Why It Matters

Tasks typically require hundreds of episodes, lasting anywhere from tens of minutes to several hours.
It heavily tests long-term planning, long-context management, and iterative debugging, rather than one-shot answering.
On average, each task consumes 9.9 million tokens, roughly 231 episodes, and 85.3 minutes.

Results

The strongest tested model achieved only 15.2% pass@1 under a partial reward threshold of 0.95.
Under a full reward threshold of 1.0, the pass rate was 10.9%.
Across all models, average pass rates were merely 4.3% and 1.7%, respectively.

The authors also analyzed failure modes and have made the benchmark openly available for future research.

More from coding & agent