Meta Introduces Muse Realtime Avatar: Speech-Driven Live Avatars at ~870ms Latency
alex_conneau · x · 2026-09-24
Meta AI Research unveils Muse Realtime Avatar, extending Muse Realtime Voice into expressive, interactive avatars with 870ms response latency, coming to Muse and Muse Charm.
Key technical points:
- Voice and avatar share a single speech-token (VQ) stream: the audio decoder turns tokens into speech while the avatar model consumes the same stream, keeping voice, lip motion, and expressions synchronized.
- The avatar is an audio-driven Diffusion Transformer conditioned on speech tokens, reference media, and a rolling window of recent video latents, generating video in short causal chunks that carry appearance and mannerisms forward indefinitely.
- Beyond talking heads: full-body illustrations gesture as they speak, and animals or everyday objects become expressive without losing their distinctive character.
Related event: Meta Unveils Muse Realtime Avatar: Sub-second Realtime Digital Humans(18 posts)→
More from Models
- antirez backs low-bit DeepSeek 4.1 Flash over Qwen 27B dense on Apple M5 Macs — antirez · 2026-11-12
- Gemini re-renders images as text when vision is removed, dropping only 6% WSR — zacharynado · 2026-11-12
- Bare-harness coding benchmark: Fable 5.2 best overall, Gemini 4 Carbon best value — Semantic_meaning · 2026-11-12
- Google's Physical-Engineering's Last Exam: all 14 frontier models score below 50% — dl_weekly · 2026-11-12
- Reddit's 2026 LLM hall of fame: Opus 5.5 still unbeaten after 9 rival releases — G0dZylla · 2026-11-12
- Commit leak reveals unreleased Gemini 4 variants: Argon, Hydrogen and Helix (unconfirmed) — BlackHC · 2026-11-11