Meta's Muse Realtime Avatar research blog: audio-driven DiT brings any image to life in real time
AIatMeta · x · 2026-09-24
Meta published its research blog on Muse Realtime Avatar. Key points: (1) any reference media — portraits, illustrations, animals, objects — becomes an expressive live conversational avatar with coherent facial, hand and full-body motion; (2) Muse Realtime Voice produces a speech-token (VQ) stream carrying content and prosody, which Avatar consumes to generate synchronized video; (3) it's an audio-driven Diffusion Transformer generating causal video chunks conditioned on tokens, reference media and a rolling window of recent latents, keeping computation bounded indefinitely; (4) companion tweets cover a 2-step distilled student and blind-test wins over commercial systems.
Related event: Meta Unveils Muse Realtime Avatar: Sub-second Realtime Digital Humans(18 posts)→
More from Multimodal
- Fable 5.2 impresses with pixel art gaussian splats, posts go viral — Flomerboy · 2026-11-12
- Seedream 5.5 nails a consistent character sticker set at just $0.02 per image — azed_ai · 2026-11-12
- Ant Group's open-source Ming-3.0-tts-0.5B tops Open TTS Leaderboard with 0.88 WER — NielsRogge · 2026-11-12
- Viggle Remake recreated in ComfyUI: swap any character in your videos with MiniMax H3 — TheLocalLab · 2026-11-12
- Pika Integrates Seedream 5.5: Split Images Into 100+ Layers for Precise Multi-Turn Edits — aziz4ai · 2026-11-12
- Llama app adds simultaneous voice, image and video input, running locally on LFM3-Omni — mervenoyann · 2026-11-12