UniWorld-View tops WorldScore with controllable novel-view video and 4D scenes
量子位 · wechat · 2026-07-24
UniWorld-View tops a world-model leaderboard with occlusion-aware novel view synthesis
The article says UniWorld-View has taken the top spot on the WorldScore leaderboard. Built by 兔展智能 together with Peking University and Peng Cheng Laboratory, the model can take one image or a short video and generate a new-view video with explicit camera control — push, pull, pan, tilt, or even a 360° orbit around the scene.
What the model does
- Uses one codebase and one model for both single-image 3D generation and video 4D generation.
- The training data comes from 元空智能.
- The project’s code and weights are fully open-sourced.
- The implementation is reportedly adapted for domestic Ascend compute.
Why large-baseline novel view synthesis is hard
The piece explains that conventional NeRF / 3DGS style reconstruction needs dense viewpoints, while diffusion-only methods often treat camera pose as a weak condition and lose control on large viewpoint changes.
The key technical ideas
- Occlusion-aware point-cloud rendering:
- Double reprojection is used to find pixels that cannot “return” under a round-trip projection; these regions are accumulated into an occlusion mask and left for the generative model to fill.
- Normal filtering removes points facing away from the camera to prevent back-side leakage.
- Dual-stream conditioning:
- A geometry stream injects point-cloud renderings plus masks to anchor structure.
- An appearance stream uses a new Ref-DiT module where the new-view feature queries source-video keys/values via cross-attention, so missing textures can be borrowed from the original video.
Extending to 4D reconstruction
The article says the system can generate multiple views, then feed them into dynamic 3DGS for 4D reconstruction. It decouples time and camera motion by:
- first generating a static 360° anchor set with time frozen on the first frame,
- then generating synchronized multi-view videos from fixed viewpoints,
- and finally optimizing a 4DGS scene from the generated views.
The broader takeaway is that the team is pushing toward a single pipeline for controllable view synthesis, open-world video generation, and 4D scene reconstruction.
More from Multimodal
- Skyfall GS Uses Flux to Refine Gaussian Splatting, Accepted at ECCV 2026 — ducha_aiki · 2026-09-11
- Lumara AI Film Festival Comes to NYC Oct 26, Top AI Filmmakers to Compete — 0xAllen_ · 2026-09-11
- Pterodactyl Detective: An AI-Generated Proof-of-Concept Trailer — PterodactylDetective · 2026-09-11
- Imperium Game Trailer Showcases AI Video Generation — keaslenyt · 2026-09-11
- FLUX.2 Klein Drifts Hard on Character Expressions While Free Gemini Holds Likeness — wacomlover · 2026-09-11
- Tencent Hunyuan releases AuK code and weights on GitHub with ComfyUI and fine-tuning support — aigclink · 2026-09-11