MoviEdit: Long-form Multi-Shot Video Editing Made Easy
Suhyeon Lee, Zhengang Li, Soo Ye Kim, Peihao Zhu, Jong Chul Ye, Chun-Hao Paul Huang, Duygu Ceylan, Yan Kang, Hyeonho Jeong
cs.CV, cs.AI, cs.LG
2026-11-02
MoviEdit drafts the edit on a subsampled video, then renders it chunk by chunk. Text-guided overall success is 0.790 on a 15s multi-shot bench, against 0.764 for Aleph 2.0.
Most video editors still consume a few seconds of one continuous shot. Long multi-shot footage breaks that at the cut. Scene, camera, and appearance jump, but the edit has to stay on the same person and look like the same change. Editing each shot alone drops the link. Joint global attention over the source and the video being synthesized scales poorly, because cost grows with the square of length.
Paired supervision is missing as well. To learn that a wardrobe change survives a cut to a side view, source and target need the subject matched across shots. That labeling does not exist in the wild. Current editing benchmarks score short single shots, so they never check whether the edit hits the wrong person after a cut, or vanishes when the person returns. MoviEdit comes from Adobe and KAIST.
Three pipelines use no manual labels. They filter 205,228 candidate pairs down to 86,619, drawn from 61,251 raw clips. The abstract rounds the count to 87K. Clips are mostly 540p at 24 fps, about 10 to 15 seconds, with 4 to 6 shots. Training also mixes in a private set of 5-second single-shot pairs.
Pixel-aligned editing is the main task: the output stays on the source pixel grid and changes only what the instruction names. First frames from every shot are tiled into one grid and edited in a single pass, so the cross-shot decision is made once. A first-frame-conditioned single-shot editor then spreads each decision inside its shot. Two QA stages drop 17% and 38% of multi-camera pairs, and 32% and 47% of cinematic pairs. A localization track follows the subject with a segmenter and supervises highlight, outline, and color fill, dropping 24%. Semi-pixel-aligned pairs relax the pixel match so the pipeline can build environment, medium, and body-shape edits. Source and target share per-shot captions and initial noise. Character edits share a denoising path until a fork. Environment edits inject source edges or depth through ControlNet. This track drops 66%.
One DiT mixture-of-experts denoiser runs both passes. The text encoder and video VAE stay frozen, and the weights are shared. Initialization is a short-form editor with no multi-shot training. Full fine-tuning uses 64 H200s: 17K steps for text guidance, then 8K more for edited-frame guidance. Reference frames come from shots where the subject is present. The benchmark provides no masks.
Pass one keeps about half the frames at lower resolution and writes a bidirectional draft. That draft is the appearance anchor. Without it, autoregressive rollout drifts as the clip runs on. Pass two generates full resolution in causal chunks, each conditioned on recent outputs, the draft, and the matching source chunk. The source chunk holds pixels that should not change. The passes differ in attention mask and resolution.
Velocity moment matching distills the draft to 12 steps and rendering to 4. At 540p and 361 frames on 8 H100s, full attention takes 364.1 seconds, sliding windows 258.9, KV cache 166.8, and distillation 11.8. That is about 30.9 times faster, and shorter than the 15-second clip. Inference can take several reference frames, from any shot.
MoviEdit-Bench has 64 generated multi-shot videos, 128 scenarios, and 8 difficulty buckets. Overall edit success is strict. A sampled frame counts only if a vision-language judge marks the edit present, on the right subject, complete, and clean outside the target. Edited-frame tasks must also match the reference.
Text-guided editing at 15 seconds:
| Method | Overall↑ | Non-subject LPIPS↓ | Precision↑ | Cross-shot consistency↑ |
| MoviEdit | 0.790 | 0.0741 | 0.904 | 0.904 |
| Aleph 2.0 | 0.764 | 0.0916 | 0.886 | 0.981 |
| HappyHorse-1.0 | 0.623 | 0.246 | 0.729 | 0.894 |
| Gemini Omni Flash | 0.586 | 0.153 | 0.705 | 0.893 |
Recall is 0.861 versus 0.778 for Aleph. Subject LPIPS is 0.121 versus 0.146. Prompt alignment is 0.884 versus 0.846. Aleph leads cross-shot consistency, 0.981 to 0.904.
On the open-source side, Wan-Edit is the strongest preserver, with non-subject LPIPS 0.0816 and precision 0.840. SDEdit sits at 0.311 and 0.321. The paper reports overall success well above this group.
When the reference frame may come from any shot, overall success is 0.786 for MoviEdit, 0.781 for Aleph, and 0.486 for HappyHorse. On first-frame references, Aleph scores 0.806, MoviEdit 0.803, and GenProp 0.581. In that subset Aleph preserves unrelated pixels better, LPIPS 0.0412 versus 0.0586, while MoviEdit leads cross-shot consistency, 0.981 versus 0.906.
More steps and more pairs keep raising the curves. Training sees at most 15 seconds. Qualitative runs reach 25 seconds and 17 shots, with no separate score.
The piece other teams can reuse is the data recipe. Edit every first frame inside one grid, then propagate inside the shot. No preexisting multi-shot editor is required. Semi-pixel pairs cover shape changes that pixel-aligned construction cannot build.
Bidirectional context runs only on the thin draft. Full resolution stays chunked, and the draft is what limits appearance drift. On 8 H100s the distilled pass takes 11.8 seconds for a 15-second clip. That is enough for offline work and short of interactive preview.
Against closed models the lead is small. Text guidance is 2.6 percentage points higher, and arbitrary-frame guidance is 0.5 points higher. Aleph still wins cross-shot consistency.
Lookalike people receive the edit on the wrong body, and a partly visible target is hard to link across shots. Figure 12 shows both. The edit is also required to keep the source motion and story. Free changes to action or plot are outside this paper.
All 64 benchmark videos are generated. Semi-pixel training pairs come from multi-shot generators too, so the test distribution can sit near the training distribution. Ranks against closed models should not be read as on-set production results. Overall success is an AND over vision-language yes/no checks, and agreement with human raters is not reported. The main table has no confidence interval, so whether 0.790 versus 0.764 survives a new batch of clips is unchecked.
The open-source editors were not trained for 15-second multi-shot inputs. The gap shows that those checkpoints do not cover the task. It does not show what another architecture would score on the same pairs and the same step budget. Multi-camera footage and the short single-shot pairs are internal. The release the paper commits to is the benchmark. Scores are not split across the 8 difficulty buckets, so mistakes on crowds can disappear into the average.