Tencent Hunyuan Releases Long-Horizon Agent Benchmark
Tencent-Hunyuan · hf · 2026-07-13
Tencent Hunyuan has released Long-Horizon-Terminal-Bench, a benchmark designed to evaluate agent capabilities in long-horizon terminal tasks.
Benchmark Design
- Features 46 long-horizon tasks across 9 scenario categories.
- Includes tasks like experiment replication, software engineering, multimodal analysis, interactive gaming, and scientific computing.
- Adopts a Terminal-Bench style but breaks tasks into finer sub-steps, providing dense intermediate rewards and partial credits.
Why It Matters
- Tasks typically require hundreds of episodes, lasting anywhere from tens of minutes to several hours.
- It heavily tests long-term planning, long-context management, and iterative debugging, rather than one-shot answering.
- On average, each task consumes 9.9 million tokens, roughly 231 episodes, and 85.3 minutes.
Results
- The strongest tested model achieved only 15.2% pass@1 under a partial reward threshold of 0.95.
- Under a full reward threshold of 1.0, the pass rate was 10.9%.
- Across all models, average pass rates were merely 4.3% and 1.7%, respectively.
The authors also analyzed failure modes and have made the benchmark openly available for future research.
Related event: Tencent Hunyuan Releases Long-Horizon Terminal Bench Exposing Model Limits(6 posts)→
More from coding & agent
- Codex tip: use Sol with Astra and Luna sub-agents to save usage — pvncher · 2026-09-11
- agents-best-practices: a provider-neutral Agent Skill for designing and auditing agentic harnesses — tom_doerr · 2026-09-11
- Cognition's SWE-2 uses a KKT duality argument in RL to shift the effort Pareto curve — YouJiacheng · 2026-09-11
- First-ever Three.js Conference lands in Paris, with a panel on AI-shortened design workflows — OdinLovis · 2026-09-11
- Data engineering, not agent frameworks, is the real bottleneck for enterprise AI agents — dhruv2038 · 2026-09-11
- RTK Terminal Compression Cuts Tokens but Leaves Your AI Coding Bill Unchanged — Bartaseth · 2026-09-11