CoArena Launches: Crowdsourced Arena Benchmark for Computer-Use Agents Backed by YC
ycombinator · x · 2026-08-23
Coasty launched CoArena, billed as the biggest crowdsourced benchmark for Computer-Use agents, backed by Y Combinator.
Static suites leak into training data, so scores climb without real capability gains. An arena can't saturate: every battle is a real task from a real person, with two frontier models running side by side in identical sandboxes — every click, screenshot and step recorded. Humans judge blind, down to verdicts on individual steps.
Early findings:
- Only 66% of agent runs finish their task at all; 1 in 3 dies before any score exists
- On identical tasks, the best frontier agent completes 87% of runs; the worst only 42%
The leaderboard is live; users can watch battles, judge blind, and post their own tasks.
Related event: CoArena Launches as Largest Crowdsourced Computer-Use Benchmark(2 posts)→
More from coding & agent
- How to hook Claude Code or Codex up to iMessage in 3 steps — jacob_posel · 2026-10-16
- Are the fastest GPU kernels now written by AI? Kernels are hyper-verifiable, argues one dev — fleetwood___ · 2026-10-16
- Anthropic lead engineer's 4-question prompt makes agents more ambitious, now a free skill — daniel_mac8 · 2026-10-16
- Simon Willison builds Chirp Pong: two phones syncing game state via audio chirps, in static HTML — Simon Willison · 2026-10-16
- ChainForge dev shows 'feature map' for orchestrating subagents on large PRs — IanArawjo · 2026-10-16
- aiDotEngineer benchmarks agent orchestration pipelines, staging humans as Foreman and workers — vikvang1 · 2026-10-16