Tencent Hunyuan's E-Bench: Top LLMs Score Under 74% in Real-World Multi-Step Agent Tasks
腾讯混元 · wechat · 2026-08-03
Tencent Hunyuan, alongside Tsinghua AIR and Southeast University, introduced E-Bench, a new benchmark evaluating LLMs on multi-step tool-use capabilities within real-world product scenarios.
Benchmark Design & Core Mechanism
- Real-World Simulation: Builds fully synthetic environments based on Honor of Kings, QQ Music, and Tencent Meeting. It features 323 state-modifying tasks across 41 DB tables and over 76,000 data rows.
- Dual-Gap Design: Creates an "Information Gap" (hiding global DB states) and a "Tool Gap" (restricting models to business-level tools), forcing genuine multi-step interactions without shortcuts.
- Deterministic Eval: Uses exact database state diffs as the strict grading criteria, eliminating subjective variance from semantic judges.
Key Findings & Insights
- Reliability is the Bottleneck: Across 11 frontier LLMs, the average success rate was only 54.56%. The best model achieved 73.79% (Avg@3), but dropped to 58.82% for consistent triple-pass (Pass³) reliability.
- Code Execution Helps, But Isn't a Silver Bullet: Allowing Python code execution boosts computation-heavy tasks but offers limited gains for reasoning-heavy cross-step dependencies. The best Pass³ remains below 70%.
- Cost-Effectiveness Matters: Tencent Hunyuan Hy3 shows a strong cost-performance ratio, costing $0.053 per task ($0.029 with code) while scoring 64.40% on E-Bench-Code.
- Premature Action is Fatal: Models typically fail not due to lack of capability, but because they act before gathering complete information.
More from Models
- Anthropic Engineer: Tying Models and Harnesses Together Is Key to Peak Performance — May_F1_ · 2026-08-03
- Surge AI Exec Reveals Why Benchmarks Fail: $15M Cost and Widespread Contamination — alex_verem · 2026-08-03
- What's the difference between these 3 models? — Clair_Personality · 2026-08-03
- When can I expect distilled LoRA for MiniMax H3? — Character_Title_876 · 2026-08-03
- OpenAI's Improved Price-to-Performance Credited to Chinese LLM Pressure — haider1 · 2026-08-03
- Dev One-Shots Full 3D Game with Opus 5: Complete with Models and Music — draginol · 2026-08-03