ByteDance’s FlowMimic trains video editing without masks by mimicking image and video modalities
ByteDance-Seed · hf · 2026-07-21
ByteDance’s FlowMimic aims to unify video editing and generation without masks
FlowMimic explores a single model that can handle both generation and editing for image and video modalities. The main bottleneck it targets is data collection: existing video editing pipelines often require manual mask annotation, synthetic pair generation, and heavy VLM-based filtering, which makes the task space narrow and hard to scale.
Core ideas
- A pixel-pair temporal warped flow field that can generate paired video-editing samples in real time from image-editing samples.
- A modality mimic generation loss and editing loss to align image and video capability distributions through mutual imitation.
- For instruction-based visual editing, the model is pushed to internalize both instruction understanding and region localization.
Training design
- Adds sense-related tasks such as referring expression segmentation.
- Uses editing-region-aware latent-level loss and attention-level loss to encourage localized edits.
- The goal is to remove dependence on external helpers like extra MLLMs or mask sequences at inference time.
More from Multimodal
- Reddit user seeks ComfyUI NSFW text-to-image and image-to-video workflows under 20 GB VRAM — hobbyist2020 · 2026-07-22
- Krea 2 users recommend a two-pass Clownshark sampler setup for sharper image details — listopalafoto · 2026-07-22
- Gemini Omni Flash turns a boat cabin into a cave in Flow by Google — chrisfirst · 2026-07-22
- A simple workflow to turn a photo into an image prompt using Gemini, Grok, or GPT Image — harshitagu72595 · 2026-07-22
- A Reddit user proposes a consistency LoRA to keep anime and game scenes visually stable — ThirdWorldBoy21 · 2026-07-22
- Hand-painted figurines run through Seedance look eerily alive — cocktailpeanut · 2026-07-22