Apple and Cornell's FastFrames hits 36 ms per frame, making video generation interactive
NielsRogge · x · 2026-11-21
Apple and Cornell researchers (including Vladlen Koltun) released FastFrames, a family of autoregressive video diffusion models fast enough for real-time interaction.
- Profiling SOTA pipelines shows VAE decoding and KV cache updates — usually treated as fixed implementation details — account for over half of end-to-end inference latency; the paper replaces both with lightweight alternatives
- The 1.3B variant runs at 36 ms per frame (28 RPS) on a single B200, matching a nearly 8× slower SOTA model in quality
- The 14B variant runs at 45 ms per frame (22 RPS) on an 8×B200 node with significantly better visual quality
- Demonstrates real-time camera-conditioned and physics-conditioned video generation
Paper: arXiv:2611.23226; code and models (FastFrames-L/S) open-sourced on GitHub and Hugging Face.
More from Multimodal
- 10 Gemini prompts to edit raw footage into polished videos 10x faster — anthara_ai · 2026-11-21
- VidLab launches: an API aggregator undercutting Seedance, Veo and GPT Image official pricing — eptwts · 2026-11-21
- Sarvam AI video dubbing demo: upload, pick a language, and dub — umesh_ai · 2026-11-21
- Someone had Claude Opus 5.5 make a short film about itself — prasenx · 2026-11-21
- Sacred Geometry Mandala prompt template turns any subject into intricate geometric art — LudovicCreator · 2026-11-21
- AI-generated Winterfell scene from Game of Thrones looks strikingly real — LudovicCreator · 2026-11-21