Lip-syncing a 3:20 AI music video on one RTX 5090: LTX + Wan 2.2 workflow
keto_brain · reddit · 2026-11-05
A creator shares the full ComfyUI pipeline behind "Low Battery" — a 3:20, two-character, 42-shot AI music video rendered entirely on one RTX 5090 via ComfyUI's HTTP API, with Claude Code writing the graph-building and cutting scripts.
Core trick: make LTX lip-sync to your vocal. Encode an isolated vocal (split with Demucs) via LTXVAudioVAEEncode, freeze it with SetLatentNoiseMask(mask=0), concat with the video latent, and sample with CFG 1.0 / euler / ManualSigmas — all core nodes.
Settings that mattered:
- LTX 2.5 distilled int8 (transformer + Gemma encoder): 2x faster than 2.3
- 1280x704 @ 24fps, 8k+1 frames; audio length must exactly match frame count or mouths drift
- VAEDecodeTiled temporalsize 96 / overlap 16 to avoid 32GB VRAM OOM
- CFG=1 means negative prompts do nothing — write everything in the positive prompt
Wan 2.2 (14B I2V, Lightning LoRA 0.3, 5 steps 2+3, RIFE x2) handles acting shots: 121 frames is the sweet spot; trim frame 0; prompt action sequences not states; let Wan do the "entering" motion; pre-crop input to 1024x704; hand off a Wan frame as the next LTX shot's still. Shots are Sing (LTX), Act (Wan), or Graphic (HTML — both models garble text).
Pitfalls: "slow push-in" runs away into extreme close-ups; calm seeds win; never seed from mp4 frames (h264 blocks animate as crawling squares) — use lossless PNGs. 42 shots cost 61 LTX takes + 33 Wan takes. Total cost: 8,800 ElevenLabs credits, a Claude subscription, $1 of electricity; all models are free and open.
Related event: Creator Renders 3:20 AI Music Video Entirely on a Single RTX 5090(2 posts)→
More from coding & agent
- Gov software co-founder: 'We vibe coded a bunch of slop' — even the specs were AI-written — alex_verem · 2026-11-05
- Can't get Dots or Duet in Europe? 6 open-source clones, top at 7.2K stars — femke_plantinga · 2026-11-05
- Open-source ArionTalk demoed at Google Web AI Summit: one tag, voice AI for any website — jason_mayes · 2026-11-05
- Over 40% of GitHub pull requests are now AI-generated, up from 17% in March — marlene_zw · 2026-11-05
- Yacine flags unsolved agent training problem: how to build state-based curricula for agents — yacineMTB · 2026-11-05
- Yuppsy launches a real estate website MCP server, 3 sites free — scuty · 2026-11-05