WaveGen: codec-free end-to-end diffusion hits SOTA text-to-speech via Spectral Forcing

Ajou · hf · 2026-11-11

The Ajou team presents WaveGen (NeurIPS 2026), a truly end-to-end diffusion framework for text-to-waveform generation.

Spectral Forcing jointly learns spectral and waveform trajectories on a shared DiT backbone: shared blocks feed disentangled waveform and spectral DiT heads whose states are updated at the same diffusion time step and feed each other's next step.
It requires no external audio codec, vocoder, latent model, alignment module, or self-supervised representations, generating waveforms directly from text (with optional reference speech).
The authors report state-of-the-art text-to-speech results.

Code and models are available on the project page.

More from Multimodal