WebArena-Pro launches: 300-task multimodal web-agent benchmark across 20 apps
xhluca · x · 2026-11-16
Researchers from McGill/Mila and collaborators released WebArena-Pro, accepted to NeurIPS 2026 (E&D Track).
- 300 tasks across 20 self-hosted applications, spanning productivity, communication, analytics, business operations, design & media, and travel.
- Multimodal: agents must read text, interpret images, listen to audio and watch video to complete tasks.
- Reproducible: each task runs in an isolated environment.
- Ships with a public leaderboard (50-step success rates, overall and per-domain), plus code on GitHub and data on Hugging Face.
More from coding & agent
- PyBot: 1,000 agents wrote a 600k-line rule-based chatbot that matches SmolLM — lvwerra · 2026-11-16
- Self-improving agent papers this week: agents learn the benchmark, not the task — TheTuringPost · 2026-11-16
- 1,000 agents, one week, 630k lines: HF builds a weights-free Python chatbot — _lewtun · 2026-11-16
- One-person content agency targets $30K/mo with 11 AI agents — full playbook — VibeMarketer_ · 2026-11-16
- Hugging Face releases definitive fine-tuning guide for 3D generation model TRELLIS.2 — ariG23498 · 2026-11-16
- WebArena-Pro: a 300-task multimodal benchmark across 20 self-hosted apps for web agents — McGill-NLP · 2026-11-16