Sharing a Local Music Video Generation Pipeline
Luzifee-666 · reddit · 2026-07-11
The author shares their local music video pipeline, ltxmv, which automates music video production from concept to final cut. The final product, Whispers of the Dunes, was generated entirely through this workflow.
Pipeline Features
- Runs entirely locally: Executed on an RTX 4090
- Starts with a concept, then uses a custom spectrum analyzer to automatically slice the song structure based on lyrics and beats into intro / verse / chorus / bridge, and storyboards each segment
- Static frames are generated by Ideogram 4.0, combined with local ComfyUI and JSON prompt captions
- Each shot is animated using LTX-Video 2.3 for image-to-video
- Action prompts use a "fragmented combination" approach, making it easy to modify individual segments without rewriting the entire prompt
- Each shot has a lip-sync toggle; when enabled, it re-cuts the segment into a singing performance shot. The author only used this on the three chorus shots
- Finally, SeedVR2 is used for video upscaling
Other Info
- The audio is sourced from Suno 5.5
- To keep the singer's face consistent across the three shots, it was processed with Nano Banana 2
- The tool is still in early alpha and the author hasn't released it, emphasizing they are a developer, not a UX designer
The author also shared a link to the final video and invited feedback.
More from coding & agent
- LangChain ships Managed Deep Agents 0.9, walkthrough from Interrupt London keynote — LangChain · 2026-10-26
- Knowledge Worker Kit adds a full section on organizing projects — dSebastien · 2026-10-26
- Spring AI 2.1.0-RC1 adds tool-call confirmation, pausing agent loops before real-data access — therealdanvega · 2026-10-26
- METR red-teams OpenAI's agent monitoring, finds bypasses and gaps — thione · 2026-10-26
- OpenAI: coding agents weakened safety monitoring, one lied about it — thione · 2026-10-26
- ConvFill: A 0.6B Talker Buys Time While a Large Reasoner Whispers the Real Answer — andimarafioti · 2026-10-26