Jev + WebMCP solves 100% of benchmark tasks at 112x lower cost than GPT-6 Astra
QuanquanGu · x · 2026-09-19
The WebMCP benchmark team reports that Jev paired with Mercury 2.5, a fast low-cost LLM, solved all 49 browser tasks at roughly 112x lower model cost than GPT-6 Astra with code execution, and 245x lower than Astra using screenshot-based computer use.
Jev's raw browser-control accuracy was unremarkable — only 25/49 tasks solved. Adding WebMCP nearly doubled that to 49/49 while cutting costs further. The team used Browser Use's open-source Ultrafast harness with reliability improvements.
Aditya Grover suspects Jev itself is a diffusion LLM: generating structured outputs like JSON in parallel resembles infilling while sampling from a dLLM, which may explain the strong showing.
More from coding & agent
- Hindsight ships 0.8B model for agent memory decisions: 2.5x cheaper, 1.9x faster — victor_explore · 2026-11-04
- Human Oversight Is 70-75% of AI Agent Costs in Banking, McKinsey Finds — mikeflache · 2026-11-04
- Over 80% of Major Financial Institutions Now Run Agentic AI in Production — mikeflache · 2026-11-04
- Mnemos MCP nears launch after shared memory layer fix — RileyRalmuto · 2026-11-04
- Moving mobile dev to remote machines makes it multiplayer across product, design, engineering — rudrank · 2026-11-04
- Kuaishou's EvoRec: self-evolving agentic recommender lifts ad revenue 4.6% — _reachsumit · 2026-11-04