LEGO: lifting-free exocentric-to-egocentric video generation beats depth-lifting SOTA pipelines

25frms · hf · 2026-10-09

Task: generating egocentric video from a single exocentric recording—little view overlap, mostly unobserved target view.
Unlike SOTA explicit pipelines (depth → point cloud lifting → reprojection, where depth errors become misplaced content), LEGO fine-tunes an LVSM-style view synthesizer to render the egocentric view directly, resolving cross-view correspondence internally.
Key argument: its probabilistic mapping trades fine texture for structural alignment—ideal for a diffusion generator whose denoising restores detail; per-region confidence masks low-confidence areas and guides early denoising steps.
Consistently outperforms the explicit SOTA and generalizes to other datasets without retraining.

More from Multimodal