Tencent Hunyuan's Prism: dynamic sparse attention speeds up 2K joint video-audio training 2.5x

Tencent-Hunyuan · hf · 2026-10-06

Tencent Hunyuan introduces Prism, a dynamic sparse attention framework for natively training joint video-audio generation models at 2K resolution.

Problem: full attention's quadratic cost balloons at high resolution, and attention spreads over redundant tokens, diluting learning signals; existing sparse attention targets training-free speedups or ignores that cross-modal video-audio interactions concentrate around sound-producing regions.
Method: Prism organizes tokens into spatiotemporal macro-zones, estimates each zone's information structure via channel-wise video feature variance and audio-to-video cross-attention norms, dynamically assigns tailored block shapes per zone (finer partitioning where visual content varies fast and audio-visual coupling is strong), and uses hybrid block selection for per-query sparsity.
Results: 2.5x training speedup over full attention while surpassing it in generation quality.

More from Multimodal