Open agent eval suite for BrowserGym, Harbor and MCP ships with an easy CLI
xhluca · x · 2026-11-17
A multi-institution collaboration announced that code for running agent evaluations on BrowserGym, Harbor, and MCP is already available.
Highlights:
- Eval code is open-sourced and ready to use now
- Co-author imenelkar built a CLI that makes managing evals much easier
- Large author team spanning many known researchers
A ready-made infrastructure for teams evaluating browser/tool-calling agents.
More from coding & agent
- Baseten launches Voice Agent Stack; Realtime Voice hits 73.5% on τ²-Bench at ~$0.01/min — baseten · 2026-11-17
- Dev building a self-hosted MCP gateway to centralize auth and deployment — hakku276 · 2026-11-17
- Hala MCP + Claude workflow generates pro-grade motion graphics videos from prompts — aziz4ai · 2026-11-17
- WebArena-Pro launches: 400 tasks across 20 real websites for browser agents — xhluca · 2026-11-17
- GTM agent forgets 4-month-old conversation and double-replies to a lead — Common_Dream9420 · 2026-11-17
- Recall.ai raises $38M Series B, launches API to send agents into video meetings — ycombinator · 2026-11-17