Meta's real-time avatars: distillation recipe cuts 120 model evals to 2, a 60x speedup for infinite video

altryne · x · 2026-11-07

altryne and SecondThread discussed how Meta's real-time avatars work, with altryne correcting his earlier claim that a student model is trained per user — all real-time avatars share the distilled model, conditioned on the user's Muse images.
The attached tech doc details the Real-Time Infinite Video Generation approach:
Starting from a high-quality bidirectional teacher, they distill a causal student with fixed-length KV cache via self-forcing + distribution matching distillation; self-forcing trains the student on its own generated context to resist error drift over long streams.
The teacher needs 40 diffusion steps with three-way CFG, i.e. 120 model evals per chunk; the tuned recipe jointly distills the diffusion process and CFG effect into an unguided two-step student needing only 2 evals — a 60x reduction while preserving teacher quality.
This tackles live streaming's twin challenges: fast enough generation for real-time interaction, and visual consistency without error accumulation.

More from Multimodal