Coarena Launches Crowdsourced Benchmark to Fix Computer-Use Data Leakage
ycombinator · x · 2026-08-23
Coasty has launched Coarena, a crowdsourced benchmark for Computer-Use agents designed to solve the problem of static benchmarks leaking into training data.
Key Features:
- Every battle is a real task submitted by a user.
- Two frontier models run side-by-side in identical sandboxes, with every click and screenshot recorded.
- Human judges blindly pick the winner before the models are revealed.
Early Findings:
- Only 66% of agent runs complete their task at all.
- The best frontier agent completes 87% of runs on identical tasks, while the worst completes 42%.
- One major frontier model frequently times out on routine operations.
Related event: CoArena Launches as Largest Crowdsourced Computer-Use Benchmark(2 posts)→
More from coding & agent
- Viewpoint: Understanding why MCP survives Skills reveals the AI landscape — cocktailpeanut · 2026-08-23
- Splitting Cloud Routing from Local Execution to Stop Agent Hallucinations in Mac Builds — NathanWilbanks_ · 2026-08-23
- Iridium MCP Server Connects Agents to Fitness Data — modelcontextprotocol · 2026-08-23
- DM Drugstore MCP Connector: Real-time Product Data — modelcontextprotocol · 2026-08-23
- Tracking complex bio-assay pipelines with Supabase, Vercel, and AI — iskander · 2026-08-23
- Armin Ronacher: LLMs make hard languages like Rust and Zig rise again — mitsuhiko · 2026-08-23