Browser MCP benchmark: 176k-565k tokens per bug fix across 5 tools

VisionInBinary · reddit · 2026-11-20

A small self-run benchmark compared 5 browser MCP tools fixing 8 agent-written bugs with Claude Code on Sonnet, 3 runs each. Median input tokens per fixed bug: ZYNLEX 176k, OpenCode Browse 290k, Playwright MCP 511k, agent-browser 541k, DevTools MCP 565k.

Key insight: the two cheapest tools both expose a 'run a script' tool, and turns dominate cost because the client resends the whole conversation every turn — ZYNLEX's own replies were only a few hundred characters per bug. The author admits the benchmark exposed bugs in their own tool (reusing a tab still emulating a phone), moving the blind number from 222k to 176k. All data is public in the repo.

Original post →

More from coding & agent

coding & agent channel →