CoSTA Decouples Speech Token Spaces to Speed Up Spoken Dialogue Models
CASIA · hf · 2026-10-30
CoSTA, a paper from CASIA and collaborators, addresses inefficiency in spoken dialogue models (SDMs): sharing one speech-token space between generation and synthesis yields long autoregressive sequences and costly inference, while swapping in lower-rate tokenizers introduces compression–quality trade-offs and expensive replacement.
CoSTA is a compatibility-preserving post-hoc framework that decouples the generation and synthesis token spaces: an adapted SDM outputs shorter compressed speech tokens, which a deterministic expansion module losslessly restores for the original speech decoder. This preserves quality and decoder compatibility while substantially shortening the autoregressive sequence and speeding up inference.
More from Research
- Investor's Last Exam Benchmark: Top AI Models Ace Research but Fail at Investment Judgment — Huangfei Jiang · 2026-10-30
- SpecSlot Boosts Pixel-Space Image Generation: +2.84 FID on ImageNet-512 at Lower Cost — Jiongfan Yang · 2026-10-30
- Apple Trains UI Navigation Agents with Online RL Directly on Live iOS Apps — apple · 2026-10-30
- KAIST's L3G-World Generates Infinite Interactive Worlds in Real Time with Latent 3D Gaussian Memory — kaist-ai · 2026-10-30
- PhysInsert from Beihang and Meitu Achieves Physically Plausible Reference-Based Object Insertion — Yue Gong · 2026-10-30
- AB-Attention Speeds Up Video Diffusion Transformers with Training-Free Accurate Block-Sparse Attention — Bin Xu · 2026-10-30