TurboT2VA distills LTX-2 to 4 steps, cutting audio-video generation time by 54.67x

量子位 · wechat · 2026-09-24

Three-stage curriculum distillation: dCM warmup → sCM refinement → sCM+DMD joint optimization. Discrete consistency distillation first stabilizes a 19B student's joint denoising structure, then continuous consistency preserves the teacher's trajectories, and distribution matching (DMD) improves perceptual quality. Ablations show staged training beats direct joint training on JavisScore.
Modality-specific handling: separate diffusion timestep shifts for audio and video, per-modality loss normalization, paired latents through a joint Transformer so each modality can attend to the other.
Quantization stack: W8A8 INT8 for linear layers, SageSLA attention with INT8 Q/K and FP8 V on H20, operator fusion and text-padding removal.
Results: at 512×768, generator time drops from 50.52s to 2.51s (20x) with JavisScore improving 0.165→0.196; at 1024×1792 on one H20, 318.74s → 5.83s (54.67x). Paper: arxiv.org/abs/2608.24674, code: thu-ml/TurboDiffusion.

More from Multimodal