WebArena-Pro: top agents solve just 62.3% of 300 tasks on 20 real websites

hhsun1 · x · 2026-11-17

Researchers released WebArena-Pro, the largest and most challenging browser-agent benchmark to date: 300 tasks across 20 self-hosted web apps and 4 modalities (text, vision, audio, video), with reproducible parallel evaluation. Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5 lead at 62.3%; OpenCode with Kimi 3.1 Pro hits 60.7%; best GenericAgent configs (Gemini 3.1 Pro omni, Gemma4 31B) reach only 30.3%. Top models fail 37% of everyday web tasks.

Related event: McGill NLP Releases WebArena-Pro, Largest Multimodal Browser Agent Benchmark; Best Agent Scores 62.3%(8 posts)→

Original post →

More from coding & agent

coding & agent channel →