Qwen-Video-Edit enables instruction-based video editing by repurposing an image model

AgeNo5351 · reddit · 2026-08-24

Core Mechanism

Qwen-Video-Edit enables instruction-based video editing by adapting Qwen-Image-Edit's Transformer to operate directly on video-VAE latents.

Technical Details

• Cross-Space Mapping: Uses two tiny projection layers to bridge Wan 2.1's video latent space into the DiT's token space, allowing static video frames to be embedded exactly like images.
• Tile Arrangement: Latent frames are arranged as tiles of one large virtual image, maintaining the positional encoding treatment from the image model's pretraining.
• Training Pipeline: Fine-tuned with LoRA or full parameters on the Ditto-1M dataset (source, edited, instruction triplets), then refined with a few steps of Wan 2.2 denoising-enhancement.

Availability

Model weights, code, and a demo page are available.

Original post →

More from Multimodal

• AssemblyAI's Universal-3.6 Pro speech-to-text lands on OpenRouter at $0.30/hr — AssemblyAI · 2026-10-16
• FLUX 3 Image Klein lands on fal: box-drawn layouts, multi-reference placement, style transfer — bfl_ai · 2026-10-16
• SGLang ships Day-0 support for FLUX 3 Image Klein: 10 reference images, BF16/FP8/NVFP4 variants — ying11231 · 2026-10-16
• Krea 3 Scores 4.81 on CPI-Bench, Second Only to GPT Image 2.5 — testingcatalog · 2026-10-16
• Higgsfield teases Kling 4.0 rollout with a monkey-driven demo video — azed_ai · 2026-10-16
• Halcyon early-fusion image models match late-fusion with 275x less pretraining compute, open-sourced — davidwromero · 2026-10-16

Multimodal channel →