NVIDIA's GATOR Turns Casual Photos into Simulation-Ready 3D Assets with Agentic Refinement
kwangmoo_yi · x · 2026-10-21
NVIDIA researchers (Qirui Wu, Stan Birchfield, Bowen Wen et al.) present GATOR, a system that reconstructs complete, simulation-ready 3D objects and room-scale scenes from casual images.
Method
- Generation: pose-aware local attention mixes patch-aligned RGB, target-mask and pointmap features to condition sparse structure generation; category text guides structure while object descriptions guide geometry, textures, and physically based materials.
- Agentic refinement: an agent runs an edit-render-review loop in Blender, using original images as evidence to repair structure, topology, and textures while preserving reliable geometry and pose—no retraining needed.
- Supports room-scale reconstruction and physics simulation with humanoid robots.
Benchmarks against generation-only Pixal3D and a GPT-6 Astra agent working from scratch show GATOR is faster and more complete. Code coming soon.
Related event: NVIDIA and KAIST unveil GATOR: casual photos to simulatable 3D objects(2 posts)→
More from Multimodal
- Creator uses refmods with reference images for consistent AI character scenes — Thorozar · 2026-10-21
- AI-generated 'Optimus as Uber driver' video imagines robots delivering food — texasnewf · 2026-10-21
- Seedance 2.5 + Midjourney Prompts for Epic Fight Scenes — techhalla · 2026-10-21
- Striking 'Skull Face' AI portraits blend smoke and gold-leaf textures — LudovicCreator · 2026-10-21
- ChatGPT Slides ships full image editing: crop, mask, restyle and background removal in-app — neelajj · 2026-10-21
- AI filmmaker Dave Clark demos Promise's AI-hybrid graybox filmmaking workflow — DavidmComfort · 2026-10-21