Video-Arena: Game-Based Video Benchmark Shows Native Models Score Just 13.2 F1
Kuan Zhang · hf · 2026-10-14
A team from miHoYo AI, Tsinghua, UCAS and HKU introduces Video-Arena, a scalable agentic video benchmark that turns competitive games into verifiable tests of spatiotemporal world-state understanding and tracking.
- Three task categories: perceiving critical events, interpreting state updates, and tracking resulting states
- Videos and aligned labels are generated at scale from game records; a golden set spans 180 tasks across 9 games
- Two protocols: best native visual understanding reaches only 13.2 F1, tool-assisted agentic analysis reaches 51.0
- Native models struggle to perceive events reliably; tool-assisted systems recover events better but still fail to interpret state updates and track them consistently over time
More from Research
- Fingerprinting Leaderboard benchmarks model provenance as copying takes minutes vs millions to train — omarsar0 · 2026-10-14
- Is today's agent memory stack just a workaround for model architecture? — Sun-Tzu-Maybe · 2026-10-14
- Persona experiments on Talkie, a 13B LLM trained only on pre-1931 text — dhadfieldmenell · 2026-10-14
- HKU and Tencent open-source Multics, a multi-player world model that trains on 4 players but scales to 10 — Scobleizer · 2026-10-14
- SAP paper teaches LLM agents to predict their own failure (AUROC 0.85) — SAP · 2026-10-14
- CUHK's Orchestra robot framework completes 7.35-step subtask sequences — CUOHK · 2026-10-14