WebArena-Pro launches: 400 tasks across 20 real websites for browser agents

xhluca · x · 2026-11-17

The McGill NLP team released WebArena-Pro, addressing the gap that well-known benchmarks like WebArena, VisualWebArena, and WorkArena cover at most 5 websites, leaving out many common ones. It adds 20 popular sites including Airbnb, Google Flights, and Dropbox, spanning bookings, video, productivity, and more.

The benchmark includes 400 tasks (100 dev / 300 test), multimodal interactions, and tiered DAG evaluation, and is self-hostable and reproducible. Evals can run via BrowserGym/AgentLab, Harbor, or an external agent connected to a browser/media MCP service, with a CLI that makes managing evals easy.

Related event: McGill NLP releases WebArena-Pro browser agent benchmark(4 posts)→

Original post →

More from coding & agent

coding & agent channel →