Coarena Launches Crowdsourced Benchmark to Fix Computer-Use Data Leakage
ycombinator · x · 2026-08-23
Coasty has launched Coarena, a crowdsourced benchmark for Computer-Use agents designed to solve the problem of static benchmarks leaking into training data.
Key Features:
- Every battle is a real task submitted by a user.
- Two frontier models run side-by-side in identical sandboxes, with every click and screenshot recorded.
- Human judges blindly pick the winner before the models are revealed.
Early Findings:
- Only 66% of agent runs complete their task at all.
- The best frontier agent completes 87% of runs on identical tasks, while the worst completes 42%.
- One major frontier model frequently times out on routine operations.
Related event: CoArena Launches as Largest Crowdsourced Computer-Use Benchmark(2 posts)→
More from coding & agent
- Vercel Goes Full Backend: Python GA, Containers, and Daemons — cramforce · 2026-10-16
- WorldMemArena Accepted at NeurIPS: Memory Storage ≠ Memory Use — xwang_lk · 2026-10-16
- Vercel Ship Demos Container Deploys for Go, Ruby, Java and More — edwin · 2026-10-16
- TRL adds day-0 support for fine-tuning Muse Spark Open's 816B MoE with LoRA on 64 H100s — QGallouedec · 2026-10-16
- Databricks engineers land impactful PRs on day 2 with AI dev tools, vs week 3 before — Yuchenj_UW · 2026-10-16
- Perplexity open-sources SPACE, the sandbox platform powering its Computer agents — AravSrinivas · 2026-10-16