100 live web tasks benchmark: browser agents compared on speed and cost
ldrx · reddit · 2026-10-30
The author built 5dive, an open-source runtime for running coding agents against a headed Chrome with persistent logins, and benchmarked five browser-agent setups on 100 hard Online-Mind2Web tasks, all using Claude Haiku 5.5:
- openclaw: 35/100, median 141s, $4.80
- 5dive (claude code): 32/100, 105s, $1.39
- claude in chrome: 29/100, 67s, $0.91 (subscription est.) — fastest, clicks immediately
- 5dive (pi): 29/100, $0.65 — cheapest
- hermes: 18/100, 86s, $0.72
Top pass rates weren't statistically significant in paired tests; speed and cost were. Four simpler controlled tasks (forms, renamed buttons, saving drafts) all passed 12/12, with per-1,000-task costs from $0.86 (5dive pi) to $6.92 (openclaw). 5dive adds human takeover for logins/CAPTCHAs and approval gates for payments, installable in one command on Debian/Ubuntu. Limitations: one model, one attempt, custom judge — scores aren't comparable to the original leaderboard.
More from coding & agent
- Do you need an AI QA agent writing Playwright tests when AI already reviews PRs? — Infinite-Fly-2756 · 2026-10-30
- Octoverse: Claude Code tops fastest-growing repos; helium lands at #9 — uwukko · 2026-10-30
- Agentic MRI review takes 9th place out of 6299 teams in Kaggle RSNA knee abnormality detection — kagglingdieter · 2026-10-30
- Open-source Row-Bot demo: autonomous multi-turn agent finishes shop launch in 3 turns — Acceptable-Object390 · 2026-10-30
- Model choice is the least important decision in an AI project — alex_verem · 2026-10-30
- CapCut GPT in ChatGPT turns a prompt into a finished short video with script and voiceover in 5 minutes — thisguyknowsai · 2026-10-30