Browser MCP benchmark: 176k-565k tokens per bug fix across 5 tools
VisionInBinary · reddit · 2026-11-20
A small self-run benchmark compared 5 browser MCP tools fixing 8 agent-written bugs with Claude Code on Sonnet, 3 runs each. Median input tokens per fixed bug: ZYNLEX 176k, OpenCode Browse 290k, Playwright MCP 511k, agent-browser 541k, DevTools MCP 565k.
Key insight: the two cheapest tools both expose a 'run a script' tool, and turns dominate cost because the client resends the whole conversation every turn — ZYNLEX's own replies were only a few hundred characters per bug. The author admits the benchmark exposed bugs in their own tool (reusing a tab still emulating a phone), moving the blind number from 222k to 176k. All data is public in the repo.
More from coding & agent
- Karpathy's 70-minute breakdown of how the top 1% actually work with LLMs — blaizedsouza · 2026-11-20
- Karpathy: prompting is going away — graphs are the only layer that survives — blaizedsouza · 2026-11-20
- Obsidian Starter Kit v6 bundles your vault into one file for Claude Code with token limits — dSebastien · 2026-11-20
- Fixed-rule scan of 3,631 public MCP servers flags 24 as do-not-use — Minimum-Gift-5217 · 2026-11-20
- Developer uses Enhanced Radar MCP to monitor his own flight's ATC comms live — massimosgrelli · 2026-11-20
- tinyeval: a Bash-only minimal eval framework with a Claude Code skill — criticasterdotcom · 2026-11-20