From Synthesis to Removal: Physics-Grounded Reflection Simulation and Diffusion-Based Video Dereflection
Zepeng Wang, Jiagao Hu, Fuhao Li, Yuxuan Chen, Fei Wang, Daiguo Zhou
cs.CV, cs.AI, eess.IV
2026-08-12
A closed-loop framework from Xiaomi pairs physics-grounded reflection synthesis with a diffusion-based removal model and a benchmark, reaching state-of-the-art video dereflection at PSNR 28.84 and 87ms per frame.
Video shot through glass (shop windows, car windows, museum cases) almost always picks up a reflection layer that degrades quality and interferes with downstream vision tasks. Single-image reflection removal is well studied, but video reflection removal is largely unexplored: there are no paired training data (reflection-free vs. reflected clips of the same scene), no models that stay temporally coherent, and no dedicated benchmark.
This paper fills all three gaps with one closed-loop framework: synthesize paired reflected data, train a diffusion-based removal model, and build a benchmark to evaluate it.
The synthesis pipeline, S2R-Synthesis, performs physics-grounded augmentation in the structure space (lineart conditions) and then renders the reflected video with a trained video diffusion renderer. Its PGA augmentation module models three glass effects: roughness-induced blur (Gaussian convolution, σ∈[5,15]), thickness-induced ghosting (an added shifted copy, offset Δ∈[-50,50] pixels), and reflectance variation (affine modulation from a Fresnel decomposition).
The removal model, S2R-Removal, is built on Wan2.1-Fun-1.3B-Inp in a video-inpainting paradigm with two stages. Stage I does reflection-aware latent adaptation (LoRA fine-tuning that learns both a diffusion loss and a reflection-intensity head). Stage II does one-step pixel-geometric refinement: a single denoising step plus pixel-reconstruction, SSIM, and depth-consistency losses (using a frozen LeReS depth estimator, computed only over reflection regions). The single-step denoising is what lets it pull ahead on speed.
The S2R-Bench benchmark has two halves: S2R-Ref (60 paired videos with full-reference metrics) and S2R-Real (50 in-the-wild reflection videos scored by human perception).
The method is state of the art on both S2R-Bench and several public image benchmarks, and it is faster than the next-best method.
| Method | S2R-Ref PSNR | TC | Avg PSNR | Speed (ms/frame) |
| GenSIRR | 27.04 | 0.9648 | 28.03 | 7214.94 |
| DAI | — | — | 27.35 | 217.27 |
| RDNet | 24.71 | 0.9564 | 26.63 | 145.13 |
| S2R (ours) | 28.84 | 0.9740 | 28.77 | 87.09 |
On S2R-Ref the PSNR of 28.84 and temporal consistency of 0.9740 are both the highest, and human scores on real videos (removal 0.787, preservation 0.980) also lead. Inference costs 87.09 ms per frame, about 1.67x faster than the next-best RDNet (145 ms). The generative method GenSIRR is close in quality but roughly 80x slower.
The synthesized data is effective on its own: on OpenRR-1k, RDNet trained with this paper's synthetic data beats the base-data-only version by +4.31 PSNR.
For video enhancement, autonomous-driving perception, and surveillance, all of which suffer from glass reflections, this is a leading choice on both quality and speed, and a rare case of a diffusion model that is both fast and accurate. The synthesis pipeline also works standalone as a data-augmentation tool: if your own removal model lacks training data, you can use S2R-Synthesis to generate paired samples and feed them in.
The authors note two. Reflection is modeled as a single layer, so complex stacked reflections from multiple glass interfaces are not handled. Performance degrades when camera motion and reflection motion are strongly correlated or parallax dominates, because the model assumes the two are relatively independent.
Beyond that, the PSNR and SSIM numbers are measured mostly on synthetic pairs; whether a real-world reflection is actually removed rests on human scores from only 50 clips, which is statistically thin.