VA-Bench: Models Near-Perfect at Localizing Targets, Only ~50% Full-Task Success
jiqizhixin · x · 2026-10-12
VA-Bench (Vision-Action Benchmark) measures embodied spatial intelligence and exposes a sharp execution gap.
- Target localization: best models hit 100%; manipulation-semantic understanding: 99.6%–100%.
- Yet full-task success is only about half: Qwen3.8-max 53.93%, Opus-5 52.86%, GPT-5.6-sol 51.55%.
- Sub-task completion reaches 65.9%–68.6%, but closing the loop collapses.
VA-Bench places spatial intelligence in a full Observe → Reason → Act → Revise loop, arguing true physical spatial intelligence should not stop at answering a predefined spatial question. The takeaway: "understanding" and "execution" are not the same — models can identify the goal and parse the instruction, yet fail when they must perceive and move.
More from Embodied
- Shenzhen robot map: 36 companies, most humanoid robot firms of any Chinese city — FinanceYF5 · 2026-10-12
- Hangzhou robot map: 25 companies led by Unitree, fastest productization in China — FinanceYF5 · 2026-10-12
- Shanghai robot map: 25 companies, industrial heavyweights FANUC, ABB, KUKA lead — FinanceYF5 · 2026-10-12
- Silicon Valley hosts 67 robot companies vs Shenzhen's 36: a 5-map comparison — FinanceYF5 · 2026-10-12
- EmbodiedCLBench: First Benchmark for Continual Learning in Embodied Agents — PKU-VaLuE-Lab · 2026-10-12
- Hive Robotics trains Unitree G1 to restock supermarket shelves, autonomous by 2027 — Distinct-Question-16 · 2026-10-12