Viggle-Animate open-weights: video character replacement from one repainted frame, no pose or mask pipeline
cocktailpeanut · x · 2026-09-05
- Viggle released its first open-weight model, Viggle-Animate, a 33.1B full finetune of MiniMax-H3's ref2va transformer, jointly distilled with DMD down to three forward passes — 5s of video in 26s on one GPU.
- Usage is minimal: repaint one frame of the driving video with a new character in any image editor; the model propagates the edit across the shot, leaving motion, camera and timing untouched.
- The key innovation is dropping the traditional pipeline entirely: no pose skeletons, segmentation masks, face tracking, identity encoders, or text prompts. Since the reference is a frame of the clip itself, pose, camera, framing and lighting already align with the footage.
- It's strongest where replacement is hardest: fast motion like whipping heads and full kicks, tracked frame by frame rather than smeared.
- Commenters highlight this as a workflow "unbundling" trend: offload pose detection and segmentation to an external image editor for a single frame, making the video model itself fast — a workflow innovation implemented as a finetune.
Related event: Viggle Open-Sources Video Character Replacement Model Viggle-Animate(2 posts)→
More from Multimodal
- Voxtral 3 ships a LiveKit-based voice agent SDK streaming listen-think-speak — MistralAI · 2026-10-21
- Voxtral Mini TTS 3 hits SOTA expressiveness in 35+ languages with tag-based control — MistralAI · 2026-10-21
- Mistral launches Voxtral 3 speech models and new Voice Agents API — MistralAI · 2026-10-21
- Open-sourced pixel art normals generator ships as Rust CLI and wasm library — HugoDuprez · 2026-10-20
- Fibo 2.0 is strongest for ecommerce and creator content, weakest at UIUX design — ArtificialAnlys · 2026-10-20
- Fibo 2.0 leads old Fibo on all 9 capabilities, best at reasoning and anatomy — ArtificialAnlys · 2026-10-20