Sung-vocal lip-sync is still broken, so this dev built a local pipeline and launched a blind test
vveerrgg · reddit · 2026-10-15
A developer argues existing audio-driven lip-sync tools handle speech fine but fall apart on sung vocals: mouths lag or gape into an "O" on every vowel. He built a fully local pipeline — one workstation, commercially licensed models, no SaaS — that works from actual lyrics and their timing rather than just the audio waveform.
He's now collecting fresh eyes for a blind test: 17 pairs of clips, about 5 minutes, phone-friendly, no sign-up or IP logging. Viewers pick which mouth matches the singing and rate each clip 1-5. Full write-up of the method will follow once results are in.
More from Multimodal
- Meshy offers Hermes users a month of Pro with API access for $1 — Teknium · 2026-10-23
- Customuse launches Smart Mesh AI with geometry-aware adaptive character clothing tool — SimplyAnnisa · 2026-10-23
- AI-generated movie poster 'The Chase' crams a wolf, a plane and muscle cars into one chase — Aggravating-Sir-115 · 2026-10-23
- SteerTTS-edge: an 11.84MB steerable TTS with unchanged WER running 19x realtime on CPU — Working_Hat5120 · 2026-10-23
- ElevenLabs' Chimera pairs Eleven v4 voice with Seedance video to make character images speak — AIwithGhotai · 2026-10-23
- Open-Source Tennis Models Plus Claude Turn Video Into a World-Class Coach — tweetsatpreet · 2026-10-23