HKU and VAST's Mira-Scene boosts single-image 3D scene reconstruction with pixel-aligned layouts
机器之心 · wechat · 2026-09-30
HKU and VAST introduce Mira-Scene, a generative 3D scene reconstruction framework that fixes the geometry, scale and layout inconsistencies of single-image scene generation.
- Core idea: instead of regressing sparse translation/rotation/scale parameters, layouts are represented as dense pixel-aligned 3D correspondences via a Canonical Coordinate Map (CCM) and Point Cloud Map (PCM), turning pose regression into an over-determined alignment problem solved with RANSAC + Umeyama.
- Co-generation: a Geometry Expert generates geometry in voxel latent space while a Layout Expert generates CCM in pixel space, coupled through shared multimodal self-attention and a shared 3D positional space.
- Two-stage training: pre-training on 1M object-centric views from 60K objects, then fine-tuning on 20K 3D-FRONT scene views. Using only 80K open-source samples, it beats SAM 3D by 39.8% in 3D IoU and 16.5% in projected 2D IoU.
- Applications: object-level editing in Blender, robotic grasping, gravity/collision simulation, and embodied agent interaction within reconstructed scenes.
Code, weights and benchmarks are open-sourced.
More from Multimodal
- Dreamy AI-generated nebula-eared creature makes the rounds among AI artists — LudovicCreator · 2026-10-24
- LoRA trained on 39 images in 40 minutes makes MOSS-Image output real transparent PNGs — gatortux · 2026-10-24
- Dev uses Sol 6.1 to generate game terrain and props, then arranges them in his editor — Dimillian · 2026-10-24
- Seedance 2.5 generates 30-second consistent AI influencer dance videos — aftahi_ai · 2026-10-24
- MultiTalk generates multi-person lip-synced conversation videos from audio, hits NeurIPS 2025 — solyarisoftware · 2026-10-24
- AI-generated Candyland video shows lollipop trees and a gummy river in stunning detail — BotLordBotflix · 2026-10-24