Running fully local AI to build a prompt-to-music-video pipeline: a deep walkthrough
julianharris · x · 2026-11-15
Former Google engineer Julian Harris published a long-form report on generating a complete music video with fully local AI from a single text prompt.
- The pipeline covers: prompt-to-music generation, stemming (splitting a song into tracks), beat detection, "prompt to edit" image generation, lip-syncing the performer to lyrics, and custom fine-tuning to add AI textures to 3D worlds.
- He frames the journey with generative AI milestones: his 2019 "prompt-to-bird" conference demo and the infamous "Will Smith eating spaghetti" era, noting what was science fiction 5-7 years ago now runs locally.
- Key takeaway: the workflow is far more involved than "prompt => music => video," and the article details tool choices and pain points throughout.
More from coding & agent
- Database Architect: a Claude Code agent that designs schemas before you code — adnan_hashmi · 2026-11-15
- Teknium says his Hermes agents are now far more effective than Claude Code — Teknium · 2026-11-15
- Adobe hands CEO role to Anil Chakravarthy, previews local MCP for Photoshop, Illustrator, Premiere — shashib · 2026-11-15
- CodeClimb: a coding agent whose reviewer keeps or reverts GraphRAG tuning experiments — blaizedsouza · 2026-11-15
- Dev ships first Steam game after 15 years, built on custom Rust engine with Claude — schiehll · 2026-11-15
- Open-source flight search MCP tool flags every split ticket as unprotected for agents — Fast_Entrance7839 · 2026-11-15