WebArena-Pro: top agents solve just 62.3% of 300 tasks on 20 real websites
hhsun1 · x · 2026-11-17
Researchers released WebArena-Pro, the largest and most challenging browser-agent benchmark to date: 300 tasks across 20 self-hosted web apps and 4 modalities (text, vision, audio, video), with reproducible parallel evaluation. Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5 lead at 62.3%; OpenCode with Kimi 3.1 Pro hits 60.7%; best GenericAgent configs (Gemini 3.1 Pro omni, Gemma4 31B) reach only 30.3%. Top models fail 37% of everyday web tasks.
More from coding & agent
- Sandbox startup speed is moot if compute sits 200ms from your infra — DanielLockyer · 2026-11-17
- Open-source sessionpipe launches to unify coding-agent sessions across any harness — Scobleizer · 2026-11-17
- ChapterPal Automates Textbook Ingestion, Nears 600 AI-Readable Books by Year-End — burkov · 2026-11-17
- Vercel CEO Rauch: your next users are agents, evals replace user interviews — pritisinghhhh · 2026-11-17
- dots starts testing 1Password integration so AI agents can log in for you securely — pvncher · 2026-11-17
- Merge Ships 19 New Connectors for Its Agent Handler, From Terraform to Stripe — shensi · 2026-11-17