tts-bench round 3: 91 open TTS models and 28 paid APIs benchmarked on identical prompts
UkieTechie · reddit · 2026-11-11
Reddit user UkieTechie released the third round of tts-bench, comparing TTS models with a unified Python harness and identical prompts, adding a separate board for hosted APIs.
Setup
- 91 open models on RTX 5090 32GB, RTX 3090 24GB, and Apple M4 16GB
- 28 hosted APIs on their own board, same 6 prompts, medians over days of warm calls from Eastern US
- English/Spanish/French prompts; samples, time-to-first-audio, realtime factor, UTMOS, WER, voice-cloning similarity, and API pricing
Open board highlights
- Fastest on CUDA: Vaniq-Edge, 30 ms first audio, 298x realtime on the 5090
- Top UTMOS: Maya1 (4.487), Underdog Woof (4.442), Phonon 31m (4.424), Chatterbox (4.423)
- Best cloning: LongCat-AudioDiT 3.5B (0.875), then FireRedTTS3 (0.843)
Hosted board highlights
- Fastest first speech: AssemblyAI 30 ms, Gradium 53 ms, Palabra 77 ms
- Best UTMOS: VUI Labs Luna (4.488), also best cloning similarity of 21 APIs, but 1.2 s startup
- Cheapest $9–10 per 1M chars (Async, Giggy, Fish, Murf); ElevenLabs v4 is $80
Caveats: UTMOS is an automated naturalness predictor, not human preference; WER only measures intelligibility; hosted and local boards are never ranked against each other. A blind-vote arena is available.
More from Multimodal
- Hailuo AI announces SOTA-quality model, strong across styles, open for everyone — Hailuo_AI · 2026-11-11
- Seedream 5.5 nails microvascular surgery imagery with sub-millimeter precision — aftahi_ai · 2026-11-11
- MiniMax H3-Image aces a text-heavy sign test: 4/4 generations, zero typos — Ok_annbae · 2026-11-11
- Fable 5.2 Max Turns Istanbul Into an Explorable 3D World for ~$46 — minchoi · 2026-11-11
- Claude Fable 5.2: 10 Wild One-Prompt Demos in Under 24 Hours — minchoi · 2026-11-11
- Fable 5.2 Shows Much More Musical Diversity in Video Spanning Four Decades of C-Pop — dotey · 2026-11-11