Fully local real-time talking avatar with SoulX-FlashHead + LFM3-Audio, ~850ms latency on RTX 5090
Acceptable-Cycle4645 · reddit · 2026-11-16
The author built a fully local, real-time conversational talking-head avatar combining three models, demoed in the audio.cpp v0.9.5 WebUI:
- SoulX-FlashHead: animates a portrait in real time from generated speech
- OWSM v4: transcribes voice input (ASR runs in parallel)
- LFM3-Audio: handles conversational responses and speech synthesis
Interesting detail: LFM3-Audio is called twice — its speech-to-speech path offers only one voice, so its TTS is used separately to synthesize responses in a different voice.
Performance: 850ms latency from question to the avatar speaking on an RTX 5090; video generation remains GPU-heavy (RTX 4090+ recommended); cuDNN is essential — without it inference is 2× slower and VRAM usage doubles. Still a prototype with room for latency optimization.
Related event: audio.cpp 0.9.5 Adds Fully Local Real-Time Talking Avatar Voice Chat(2 posts)→
More from Embodied
- Meta's Muse Charm surfaces: a screen-pendant AI agent device you can already sign up for — altryne · 2026-11-16
- Prototype pairs kids' code, a real robot and Snap Spectacles AR for learning — stspanho · 2026-11-16
- Andy Rubin's Genki Robotics shows first humanoid, claims it was assembled over one weekend — xmercury_one · 2026-11-16
- Gemini Robotics 2.5 debuts with 96% pick-and-pack success on Mercedes-Benz lines — dl_weekly · 2026-11-16
- DIY holographic desk buddy costs ~$70 with an ESP32 AMOLED kit and a beam splitter prism — PurzBeats · 2026-11-16
- Embodied AI robots take over night-shift parcel sorting in Shenzhen logistics hub — yogthos · 2026-11-16