Alibaba's four-generation TTS evolution: CosyVoice to Qwen-Audio-3.0-TTS
solyarisoftware · x · 2026-10-26
Alibaba's Tongyi speech team explains its TTS roadmap: progress comes from identifying and moving the real bottleneck, not just scaling. From CosyVoice to Qwen-Audio-3.0-TTS, four generations kept the same backbone while solving four different bottlenecks (per the diagram):
- I. Represent and align (CosyVoice): unsupervised acoustic units hard to predict from text → supervised semantic tokens plus flow rendering.
- II. Schedule and factor (CosyVoice 2): causal availability and information leakage → FSQ, pretrained LM, no speaker embedding in LM, chunk-aware FM.
- III. Scale and specialize (CosyVoice 3): coverage and post-training signal → multitask tokenizer, million-hour data, larger LM, DiffRO.
- IV. Coordinate (Qwen-Audio-3.0-TTS): token-only capacity and modular mismatch → 12.5-Hz tokens, hidden-state conditioning, joint and staged optimization.
Stable decomposition throughout: text and prompt → autoregressive semantic planning → flow-matching acoustic rendering → waveform.
More from Multimodal
- LanSE: using sparse encoders to evaluate multimodal model outputs — xuanalogue · 2026-10-26
- Kling 4.0 demo: Tesla Roadster takes an impossible route around the world — SimplyAnnisa · 2026-10-26
- 4K AI music video "TENDER" made with Kling, DomoAI and Seedance — Superb_Vegetable_684 · 2026-10-26
- Moss character LoRA training finally clicks in Fizgig, beating Krea results for a few dollars — derTommygun · 2026-10-26
- Claude Code rebuilt its own ComfyUI pipeline to fix prop continuity in a 2-min AI film — Odd_Definition_9724 · 2026-10-26
- Gingerbread watercolor storybook clipart prompt shared with polished sample outputs — azed_ai · 2026-10-26