Speech Agent Arena Launch: Gemini Leads Preference, Grok Leads Success Rate
ArtificialAnlys · x · 2026-08-21
ArtificialAnlys announced the Speech Agent Arena, evaluating Speech-to-Speech models via human interactions in real-world scenarios (e.g., ordering takeout). It measures both Conversational Preference (Elo) and Task Success Rate.
Key Results:
- Preference: Google Gemini 3.1 Flash Live Preview - Minimal leads at 1,046 Elo, followed by GPT-Realtime-1.5 at 1,000. Preferred models tend to respond faster (lower TTFA) and sound more natural.
- Task Success: Grok Voice Think Fast 2.0 High leads at 94.7%, followed by GPT-Realtime-2.1 High at 91.5%. Notably, Gemini's top preference score (1,046 Elo) contrasts with its 74.6% success rate, indicating that pleasant conversation does not always result in successful task completion.
Related event: Speech Agent Arena Debuts: Grok Tops Tasks, Gemini Wins Preferences(4 posts)→
More from Models
- Claude Opus 5.5 effort levels tested: low reasoning iterates 9x faster than max — tom_doerr · 2026-10-16
- OpenAI ships GPT-Live-1.1 with new voices in six languages and better accents — OpenAI · 2026-10-16
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- OpenAI ships GPT-Live-1.1: new voices in six languages, fewer confirmations — OpenAI · 2026-10-16
- Muse Spark model appears on LMArena, testable in Battle and Agent modes — arena · 2026-10-16
- Running a multi-model UI design arena: Paseo orchestrates four design drafts per model — vista8 · 2026-10-16