Agent Arena rolls out Bash Reliability, counting only agent-caused bash failures
arena · x · 2026-10-14
Arena upgraded its Bash Recovery signal to Bash Reliability: it now reads each bash call's purpose, outcome and error cause, and whether the error was fixed within the call — counting only agent-caused mistakes while excluding network/permission errors outside the model's control. Reliable first-try command-line use earns credit. On this signal, Claude Opus 4.8 (High) leads at 15.80%, with Sonnet 5 and Opus 5.5 close behind. Backfilled scores are coming to public Hugging Face datasets.
Related event: Agent Arena Update: Claude Opus 5.5 Tops Ranking as Bash Reliability Debuts(6 posts)→
More from coding & agent
- Beancat launches an AI agent for media production with transparent per-run dollar pricing — Scobleizer · 2026-10-14
- Two devs burn $2,025 in API costs building a game with Claude, screenshots show progress — zeeg · 2026-10-14
- Sketch first, code later: prototyping with tldraw before handing specs to Claude — max__drake · 2026-10-14
- Google ran Argon overnight on a 200K-line codebase: 40% faster sessions, 2 of 6 fixes shipped — rseroter · 2026-10-14
- Argon synthesizes its own voice from scratch: pure Python, no audio libraries — fofrAI · 2026-10-14
- Musk RTs: Grok Bot Multi-Agent Team Does Your Grocery Shopping in 3 Minutes — elonmusk · 2026-10-14