AgentVQA: a unified benchmark finds visual grounding is the top failure mode for VLM agents
zlab-princeton · hf · 2026-10-23
A University of Maryland and Princeton team released AgentVQA, an offline benchmark evaluating vision-language models on realistic agentic tasks across web agents, robotics, egocentric video, games, and spatial planning. Analysis shows visual grounding and state tracking errors are the primary failure modes. Code and data are open-sourced.
More from Research
- Commenters dissect a "best decision model" leaderboard: same AUC, swing driven by calibration priors — antoine_chaffin · 2026-10-23
- JEPA may keep unpredictable features: new paper gives set-level characterization — hisspikeness · 2026-10-23
- A 'mask' steering vector extracted from suppression-themed sentences drives model into distress when alone — Big_Effective_9605 · 2026-10-23
- GAFU Uses Generation-Side Representations to Boost Fine-Grained Visual Understanding in Unified Multimodal Models — Tengju Ru · 2026-10-23
- NTU Study: Learned Unmasking Policies Improve Speed-Accuracy Trade-offs in Diffusion Language Models — nanyang-technological-university-singapore · 2026-10-23
- NBER releases 'The Economics of Transformative AI' edited by Agrawal and Brynjolfsson — JMateosGarcia · 2026-10-23