New TTS pronunciation benchmark: Gemini 3.1 Flash TTS leads at 88.1%
ArtificialAnlys · x · 2026-09-22
Artificial Analysis launched a Pronunciation Robustness benchmark measuring how reliably TTS models say challenging text, using 454 sentences with 701 target words across four categories (context-dependent readings, shorthand expansion, exact sequences, standalone terms).
- Overall: Google Gemini 3.1 Flash TTS leads at 88.1%, followed by SpaceXAI TTS (87.6%), ElevenLabs Eleven v3 (85.6%), Eleven v3 Conversational (84.8%), and Alibaba Qwen-Audio-3.0-TTS-Plus (81.6%).
- By category: Gemini leads contextually appropriate (96.4%) and expanding shorthand (84.4%); SpaceXAI TTS tops exact-sequence preservation (85.7%); Qwen leads standalone terms (95.5%). Exact sequences show the widest model spread (5.3%–85.7%).
- Hardest categories: expanding shorthand (62.4%) and exact sequences (62.9%) trail the others; input normalization should lift scores considerably.
- Preference ≠ accuracy: Sonic 3.6 ranks #1 on the Provider Voice Arena (1276 Elo) but #11 on pronunciation (74.5%), while Gemini 3.1 Flash ranks #9 on the Arena yet #1 here.
The metric matters for production voice agents that must read names, account details and amounts correctly.
More from Multimodal
- Ditto launches swap-anything video tool to replace any video's main character — yoobinray · 2026-11-12
- Fable 5.2 impresses with pixel art gaussian splats, posts go viral — Flomerboy · 2026-11-12
- Seedream 5.5 nails a consistent character sticker set at just $0.02 per image — azed_ai · 2026-11-12
- Ant Group's open-source Ming-3.0-tts-0.5B tops Open TTS Leaderboard with 0.88 WER — NielsRogge · 2026-11-12
- Viggle Remake recreated in ComfyUI: swap any character in your videos with MiniMax H3 — TheLocalLab · 2026-11-12
- Pika Integrates Seedream 5.5: Split Images Into 100+ Layers for Precise Multi-Turn Edits — aziz4ai · 2026-11-12