Building a Short Video Pipeline with Claude+ComfyUI
Available-Training-4 · reddit · 2026-07-13
The author set up a single-node canvas that strings together Claude for scripting + image generation + image-to-video + ffmpeg to create an end-to-end short video pipeline from character names to final cuts.
Why do this
He previously spent around $200 on third-party AI video services with almost zero usable footage. The problem wasn't the model itself, but that assets were scattered across multiple tools, requiring manual back-and-forth for characters, dialogue, and editing.
How this pipeline works
- Scriptwriter node: Given a theme or character name, Claude writes the script and breaks it down into scenes, including image prompts, actions, dialogue, and duration.
- Character reuse: Characters are turned into reusable "souls," maintaining consistency across scenes via avatars, turnarounds, and feature descriptions.
- Continuity fix: The author noticed the script generator would hallucinate inconsistent appearances. Feeding it a 1.5k tokens character description solved this.
- Starting frame strategy: Since i2v often starts "mid-action," a starting frame featuring the character and setting is generated first, allowing the video model to continue from there.
- Scene transitions and composition: The last frame of a scene serves as the reference for the next. Finally, ffmpeg handles splitting, transitions, and ducking the music.
- Cost control: The canvas estimates costs in real-time, and caching ensures unchanged nodes cost $0 to rerun.
The author's dilemma
He is still debating whether this use case is better suited for a custom canvas or direct ComfyUI custom nodes, and how to more reliably maintain character and scene continuity.
More from coding & agent
- ty now reads Pydantic config keywords and field metadata — charliermarsh · 2026-07-22
- Pensar Launches AI Security Agent to Autonomously Discover and Patch 0-Days — andriy_mulyar · 2026-07-22
- ty adds first-class Pydantic support, including strict and lax field handling — charliermarsh · 2026-07-22
- Google launches Gemini 3.5 Flash Cyber for CodeMender, with limited access for governments — GoogleAI · 2026-07-22
- Cursor doubles usage limits across all plans for Grok, Composer and new models — XFreeze · 2026-07-22
- Video-based proof of work is emerging as a feedback layer for coding agents — Vjeux · 2026-07-22