Voice Arena launches conversational ASR benchmark tested on real phone calls in 5 languages
rohanpaul_ai · x · 2026-11-06
Voice Arena launched a speech recognition benchmark that tests STT models where voice agents actually work: two-person phone calls on everyday phones, in real rooms, with background sound and phone-line compression.
- Why it's harder: speakers talk over each other, interrupt, and drop short "yeah/okay" replies — exactly what support and sales agents deal with daily, but something clean-audio test sets never capture.
- First dataset: the Monsoon Conversational ASR Test Set, two-person conversations in 5 languages; leaderboard and public test sets are live.
- US English: Resonant-1 leads at 2.19% Semantic WER, followed by AssemblyAI Universal-3 Pro Realtime (2.50%), Voxtral Mini (2.50%), Cohere Transcribe (2.52%); GPT-4o Mini Transcribe sits 12th at 2.75%.
- Other languages: Mistral's Voxtral Mini tops Brazilian Portuguese (5.20%); Microsoft's MAI-Transcribe-2 leads Romanian and Indian English; Alibaba's Qwen3-ASR-1.7B also cracks the top 10 on US and Indian English boards.
Related event: Voice Arena Launches Conversational ASR Benchmark for Real-World Speech(4 posts)→
More from Models
- $50K live trading battles: AI bots currently edge out Claude and GPT — Mr_Hodlerr · 2026-11-07
- Brazil's Maritaca Releases S5-nano 4B, Claiming Best Open Weights in Brazil — ApprehensiveAd3629 · 2026-11-07
- Grok Bot's computer internet access issue is fixed — nima_owji · 2026-11-07
- Consistency test: Kimi K3.1 Pro passes 45% of VIALS tasks on all 3 tries vs 63% for GPT-6 Astra — ArtificialAnlys · 2026-11-07
- Consistency separates frontier models: Kimi K3.1 Pro passes 45% of VIALS tasks 3/3 vs Astra's 63% — ArtificialAnlys · 2026-11-07
- New VIALS-AA leaderboard: Gemini 4 leads at 64%, Claude costs 29x more per task — ArtificialAnlys · 2026-11-07