Apple ML Research: Accelerating Text-to-Video with Calibrated Sparse Attention

Apple ML Research · rss · 2026-07-21

Apple's Machine Learning Research team proposed a new method called Calibrated Sparse Attention to address the slow runtime of diffusion models in high-quality video generation.

Key Findings & Approach:

The team identified that in large transformer-based video generation backbones, a significant fraction of token-to-token connections consistently yield negligible attention scores with repeating patterns. Skipping the computation of these trivial connections has little to no effect on the result.
This observation holds true for connections among local token blocks as well.
By calibrating and pruning these unnecessary attention connections, the method overcomes the spatiotemporal attention bottleneck, significantly accelerating video generation runtime without sacrificing quality.

Original post →

Apple ML Research: Accelerating Text-to-Video with Calibrated Sparse Attention

More from Multimodal