WebArena-Pro: multimodal web-agent benchmark with 20 self-hosted apps, best agent at 62.3%
xhluca · x · 2026-11-17
Researchers released WebArena-Pro, a heterogeneous, multimodal, reproducible benchmark for AI web agents, to be presented at NeurIPS 2026 in Paris.
- Sandbox spans 20 self-hosted web apps with tasks across text, vision, audio, and video (300-400 tasks)
- Public leaderboard with per-domain and step-budget breakdowns; success requires a final reward of 1
- Best agent so far reaches 62.3% success at a 50-step budget, leaving plenty of headroom
- Paper, code, and Hugging Face resources are available
More from coding & agent
- Dev shows 'constellation view' UI where every star is a live agent — kevinkern · 2026-11-17
- The constellation view: monitoring agents as stars — green, yellow, red, gray — kevinkern · 2026-11-17
- Directing AI agents by playtesting: one note turned an ugly lighthouse into a beauty — chrisfirst · 2026-11-17
- Dev directs agents with playtest notes to build a game in four days for Epic MegaJam — chrisfirst · 2026-11-17
- 943 agents, 11.5B tokens, $5,900: 97% of tokens went to re-reading context — chrisfirst · 2026-11-17
- Dev builds stunt-driving game in 4 days with 943 AI agents, 11.5B tokens and $5,900 of compute — chrisfirst · 2026-11-17