How Speech Agent Arena works: blind live voice tests, task success as the metric
ArtificialAnlys · x · 2026-10-28
Artificial Analysis explains its Speech Agent Arena methodology: humans hold blind live voice conversations with two hidden systems on the same task and pick a winner; for tool-calling tasks, Task Success Rate measures the share of eligible conversations ending with the correct final tool call. Cartesia Managed Agents hits 91.7%, up from 77.5% for Cartesia Line. The chart shows it trailing only Grok Voice 3.0/2.0 and Gemini 3.0 Live, ahead of the GPT-Realtime family.
Related event: Cartesia Managed Agents Tops Speech Agent Arena(6 posts)→
More from Models
- Claude refuses to gather legitimately public information, sparking guardrail backlash — mkheck · 2026-10-28
- DesignArena launches Code to Video Arena: models direct 15-45s films purely in code — yshan2u · 2026-10-28
- Power user teases an unreleased Grok @bot feature he calls his favorite AI moment in months — petergyang · 2026-10-28
- Will local-model progress outrun GPU depreciation? Reddit debates the buy-now window — Sensualities · 2026-10-28
- GPT-5.5 passes 46.3% of clinical tasks once, but only 28% across three runs — jonc101x · 2026-10-28
- Liquid AI's 1.7B model claimed to beat models 2-5x its size on phones and laptops — rickasaurus · 2026-10-28