Artificial Analysis Launches TTS Pronunciation Robustness Benchmark; Gemini 3.8 Flash TTS Tops at 89.5%

On Sept 22, Artificial Analysis launched a Pronunciation Robustness benchmark for TTS, with human raters judging whether English snippets (brands, place names, polysemous words) are pronounced correctly — 454 sentences and 701 target words across four challenge categories: context-dependent readings, abbreviation expansion, exact sequence preservation, and standalone terms. A comparison page blending preference Elo, speed, price, and pronunciation also went live the same day. After the Sept 23 update, Gemini 3.8 Flash TTS leads at 89.5%; sample audio followed on Sept 24.

Confirmed

• On the launch-day leaderboard, Gemini 3.1 Flash ranked first at 88.1% (later updated to 88.2%), also leading context-dependent readings (96.4%) and abbreviation expansion (84.4%) — the latter only 0.1 points above SpaceXAI TTS
• Sept 23 update: Gemini 3.8 Flash TTS tops at 89.5%, ahead of Gemini 3.1 Flash TTS (88.2%) and SpaceXAI TTS (87.6%), priced at $32.98 per million characters
• On Sept 24, Artificial Analysis posted Gemini 3.8 Flash TTS sample audio covering context-appropriate pronunciation (e.g., "St. Mary's is on Church St.") and abbreviation expansion (e.g., "Median latency was 12 ms")
• Pricing: Gemini 3.1 Flash at $18.31 per million characters; SpaceXAI TTS at $15, the cheapest model above 80%; Kokoro at $0.65, cheapest overall
• Speed: SpaceXAI TTS reaches 87.6% at 106 characters/second; Realtime TTS-2 Flash runs at 220 characters/second but only 77.8% accuracy; Kokoro is fastest but 55.5%
• The largest gap is in exact sequence preservation, ranging from 5.3% to 85.7% across models
• Preference and pronunciation diverge: Sonic 3.6 holds the highest Elo (1276) in the Provider Voice Arena but only 74.5% pronunciation robustness; Gemini 3.1 Flash's Elo is 1201
• The comparison page covers 27 of 95 models, with preference data from Speech Arena user votes
• On the same day, the full TTS leaderboard showed xAI first at 87.6%, conflicting with the launch-day Gemini 88.1% ranking

Unconfirmed

• The three result sets name different leaders; differences in evaluation scope and model sets are unexplained, and Gemini 3.8 Flash TTS is known only from the benchmark and sample audio, with no official release

Why it matters

Preference Elo only captures listening appeal and misses misread abbreviations and proper nouns; the pronunciation robustness benchmark offers a new selection criterion for accuracy-sensitive use cases like customer service and audiobooks, and combined with price and speed data enables direct cost-performance tradeoffs.

2026-09-22 ~ 2026-09-24 · 12 related posts

Full story(3 episodes)→

• Episode 1: Artificial Analysis Launches TTS Pronunciation Robustness Benchmark; Gemini 3.8 Flash TTS Tops at 89.5%(2026-09-22, 12 posts)
• Episode 2: Gemini 3.8 Flash TTS Priced at $32.98 per Million Characters, Still Slower Than Rivals(2026-09-23, 2 posts)
• Episode 3: Gemini 3.8 Flash TTS Tops All Seven Language Voice Arena Boards(2026-09-24, 2 posts)

• [source] New TTS pronunciation benchmark: Gemini 3.1 Flash TTS leads at 88.1% — ArtificialAnlys · 2026-09-22
• TTS benchmark breakdown: exact-sequence category shows 5.3%-85.7% model spread — ArtificialAnlys · 2026-09-22
• Arena Elo vs pronunciation: top-preference Sonic 3.6 scores just 74.5% accuracy — ArtificialAnlys · 2026-09-22
• TTS leaderboard: xAI hits 87.6% pronunciation accuracy, Kokoro 82M fastest at 242 chars/s — ArtificialAnlys · 2026-09-22
• TTS pronunciation benchmark pricing: Gemini leads at $18.31/1M chars, Kokoro cheapest — ArtificialAnlys · 2026-09-22
• TTS speed vs accuracy: SpaceXAI hits 87.6% at 106 chars/sec, Kokoro fastest but weakest — ArtificialAnlys · 2026-09-22
• Gemini 3.8 Flash TTS tops pronunciation robustness benchmark at 89.5% — ArtificialAnlys · 2026-09-23
• [source] TTS Price Check: Gemini 3.8 Flash TTS at $32.98/1M chars, 3x Cheaper Than Eleven v3 — ArtificialAnlys · 2026-09-23
• [source] Gemini 3.8 Flash TTS tops Artificial Analysis pronunciation benchmark at 89.5% — ArtificialAnlys · 2026-09-24
• Sample audio: Gemini 3.8 Flash TTS contextually appropriate pronunciation demos — ArtificialAnlys · 2026-09-24
• Google Gemini 3.8 Flash TTS heard in examples: shorthand expansion and contextual pronunciation — ArtificialAnlys · 2026-09-24
• Artificial Analysis launches TTS leaderboard with new Pronunciation Robustness Benchmark across 95 models — ArtificialAnlys · 2026-09-24