Sharing a Local Music Video Generation Pipeline
Luzifee-666 · reddit · 2026-07-11
The author shares their local music video pipeline, ltxmv, which automates music video production from concept to final cut. The final product, Whispers of the Dunes, was generated entirely through this workflow.
Pipeline Features
- Runs entirely locally: Executed on an RTX 4090
- Starts with a concept, then uses a custom spectrum analyzer to automatically slice the song structure based on lyrics and beats into intro / verse / chorus / bridge, and storyboards each segment
- Static frames are generated by Ideogram 4.0, combined with local ComfyUI and JSON prompt captions
- Each shot is animated using LTX-Video 2.3 for image-to-video
- Action prompts use a "fragmented combination" approach, making it easy to modify individual segments without rewriting the entire prompt
- Each shot has a lip-sync toggle; when enabled, it re-cuts the segment into a singing performance shot. The author only used this on the three chorus shots
- Finally, SeedVR2 is used for video upscaling
Other Info
- The audio is sourced from Suno 5.5
- To keep the singer's face consistent across the three shots, it was processed with Nano Banana 2
- The tool is still in early alpha and the author hasn't released it, emphasizing they are a developer, not a UX designer
The author also shared a link to the final video and invited feedback.
More from coding & agent
- Agent verified 'success' after double-voting: a lesson in harness vs model bugs — TejasKumar_ · 2026-10-26
- diginomica: agentic automation hype is overdone, and 'The Hackening' of enterprise security has begun — jonerp · 2026-10-26
- 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
- Devin launches Security suite powered by Claude Mythos with $100M+ credits — DevinAI · 2026-10-26
- HackBench injects reward hacks into agent trajectories to stress-test monitors — DanielKhashabi · 2026-10-26
- Hugging Face ships hf CLI updates: deduped cache saves 19GB, faster buckets, retryable GPU jobs — yb2698 · 2026-10-26