Claude autonomously directs an AI music video: 80 renders, ~12 GPU-hours, just 8% of a $20 plan
Cheap_Credit_3957 · reddit · 2026-11-01
Developer vrgamegirl19 used her ComfyUI "VRGDG Video Builder" to have Claude autonomously direct a Halloween-themed AI music video, with a detailed human-vs-AI breakdown:
Done by hand: giving Claude the story idea (pop-punk female rock singer, Victorian mansion party) and character descriptions; making the song in Suno 6 with stems; generating locations in Nano Banana; picking characters from Krea 2 three-panel reference sheets; setting staging rules (instruments only on stage, close-ups offstage, natural crowd); reviewing drafts and requesting reshoots; a final Topaz AI pass for blurry wide-shot faces.
Done by Claude: writing lyrics and location prompts; generating character sheets; timing every sung word with ElevenLabs Scribe and inferring who plays when from stems; planning 19 scenes (2:29) with per-shot MiniMax H3 prompts including lyric timing and facial acting; running the render queue via the Builder's Agent API — 80 renders, 12 GPU hours, mostly overnight; self-reviewing takes and reshooting failures (cloned guests, camera pulling wide on held notes); splicing fixes frame-exactly and checking lip-sync in every scene.
Settings & cost: MiniMax H3 Singularity ref2va v2 (int8), 25-step pass + 8-step 768p ref2v LoRA (4 steps, 0.35 denoise); 12 hours total, using only 8% of a $20 Claude Pro plan. The builder and a two-pass workflow JSON are open-sourced on GitHub.
More from coding & agent
- VFX Supervisor Open-Sources 20 Skills for Building an Automated Motion Design Pipeline with Claude Opus 5.5 — GCWebDesigner · 2026-11-01
- Cursor launches Annotate in design mode: give agents visual prompts by editing the page — soleio · 2026-11-01
- Open-source Garmin MCP turns training data into AI-built dashboards — indaco_dev · 2026-11-01
- GPT-5.6 Sol reportedly beats Factorio, launching a rocket autonomously — Realistic-Total-3553 · 2026-11-01
- Open-source CodeRail lets teams share one Claude Code session from any device — L_i_n_k0666 · 2026-11-01
- Dev observes AI has made systems-heavy games like RTS and strategy easier to build — IanArawjo · 2026-11-01