SQuad reduces video generation attention compute by 67x with O(N√N) complexity
jm_alexia · x · 2026-08-19
The paper introduces SQuad, a framework achieving O(N√N) complexity for Self-Attention in video generation. Using a two-stage distillation process on the Wan 2.2 5B model, it matches the quadratic teacher's quality on VBench (83.20 vs 83.08) while cutting attention FLOPs by 67x and latency by 11x per step per block.
More from Multimodal
- MLP-Splatting (ECCV 2026) code released: MLPs as light-field primitives instead of Gaussians — AjdDavison · 2026-10-16
- Digital human Fae codes her own facial expressions frame-by-frame with Claude Opus 5.5 and FACS — repligate · 2026-10-16
- Meta's Omnilingual ASR, covering 1,600 languages, named a TIME Best Invention of 2026 — AIatMeta · 2026-10-16
- ATAS: Zhejiang-led team builds world-state memory framework for consistent agentic video generation — CSProfKGD · 2026-10-16
- Flux 3 Klein's Default Templates Are Broken Without Its Prompt Upscaler — StrictMove5839 · 2026-10-16
- Kroma Touted as the Most Well-Rounded Image Model Yet on Civitai — Different_Fix_2217 · 2026-10-16