MirrorWorld: Taming Video Diffusion Models for Mirror Reflection Generation
Youjun Zhao, Alex Warren, Gary K. L. Tam, Rynson W. H. Lau
cs.CV, cs.LG
2026-08-08
Built on Wan2.1-VACE-14B, MirrorWorld adds a semantic-distillation loss for what a mirror reflects and a geometric-alignment loss for how it is arranged, beating VACE and other video-inpainting baselines on PSNR, SSIM, LPIPS and FVD.
Video diffusion models (VDMs) can synthesize high-fidelity video, but mirrors are a blind spot. The paper opens with a side-by-side: even Veo 3.1 produces mirror reflections that disagree with the scene, with wrong objects or implausible positions. The reason is that VDM training objectives contain no term requiring the mirror to match the world outside it, so the mirror region gets filled in like any other pixels.
The authors split reflection into two stacked problems: what should appear in the mirror (semantics), and where and how it should sit (geometry). Existing work is either image-level reflection synthesis or generic video inpainting; neither models the scene-to-mirror relationship explicitly. MirrorWorld separates the two.
The framework is a masked conditional video generator: input is the video with mirror regions masked out, a binary mirror mask, and a text prompt; output is the inpainted video. The base is Wan2.1-VACE-14B, fine-tuned with LoRA (rank 32). The contribution is two alignment losses added on top of the diffusion objective.
The first, Semantic Relation Distillation (SRD), handles "what". It pulls reference semantic features from a frozen foundation model (VideoMAEv2-Base) and a second set from the diffusion model's own hidden states. For each temporal slice it computes the cosine-similarity matrix between tokens outside the mirror and tokens inside, then drives the diffusion model's relation matrix toward the reference model's. The frozen model takes no gradient, so the relation is transferred one-way. This forces the mirror to contain semantically relevant content instead of arbitrary fills.
The second, Geometric Transformation Alignment (GTA), handles "how". A separate MLP head extracts geometric features and fills the mirror region with the mean of the visible region's features, so the transform estimator cannot peek at the answer. A shared regressor predicts an affine matrix from a K=5-frame temporal window, applies it through differentiable bilinear warping, and requires the warped visible features to match the mirror features in cosine distance. Using a window rather than a single frame stabilizes the geometry signal. Loss weights are λSRD=0.05 and λGTA=0.01.
The benchmark repurposes four mirror datasets (VMD-D, ZOOM, MMD, DVMD-D) into a unified format: 1,242 clips, 45,373 frames, split at the source-video level. Baselines span image-level reflection methods (MirrorFusion, MirrorVerse) and video inpainting (VideoPainter, VACE).
| Method | PSNR↑ | SSIM↑ | LPIPS↓ | FVD↓ |
| MirrorFusion | 9.51 | 0.293 | 0.699 | 513.1 |
| MirrorVerse | 9.67 | 0.312 | 0.680 | 416.1 |
| VideoPainter | 11.28 | 0.399 | 0.606 | 229.6 |
| VACE | 13.54 | 0.489 | 0.493 | 191.6 |
| MirrorWorld | 14.01 | 0.504 | 0.488 | 184.9 |
Flow-warping error (Ewarp), a temporal-consistency measure, drops from VACE's 0.026 to 0.025, with VideoPainter at 0.076. In the ablation, SRD alone (13.55) or GTA alone (13.54) trails both together (14.01), confirming the two parts are complementary.
To be honest, PSNR 14 and SSIM 0.5 are low in absolute terms; this is a problem far from solved. The claim is not that reflections are now good, but that on a unified benchmark the method is ahead of both the specialized and the general baselines.
For anyone in video generation, reflective surfaces (mirrors, glass, water) are a known weak spot with no public benchmark or method. MirrorWorld contributes three things at once: a reproducible benchmark, a decomposable method whose two losses can be toggled independently, and a clean mental model ("what" vs "how") that could transfer to any task where generated content must stay consistent with something elsewhere in the scene.
It rides on off-the-shelf Wan2.1-VACE-14B with LoRA, reproducible on four A100s.
The authors' stated core limitation: the method assumes the reflected content is at least partially visible in the frame. If the mirror reflects a person outside the camera's field of view, SRD and GTA have nothing to constrain, and the model falls back on priors to synthesize something plausible but not real. The paper's failure case is exactly a mirror reflecting an off-camera pedestrian.
Two things look under-tested. First, the ground truth is the original video, which assumes the mirror's content equals the unreflected content, but real mirrors involve left-right flips and view transforms, so the task definition is somewhat idealized. Second, pixel metrics like PSNR and SSIM applied to content that is geometrically transformed by reflection may not track whether the reflection is right in any human sense; the paper offers no subjective comparison.