NVIDIA's GATOR reconstructs textured 3D objects from casual photos with generative-agentic framework
nvidia · hf · 2026-10-13
NVIDIA and Simon Fraser University present GATOR, a generative and agentic framework that recovers complete, textured 3D object assets and their scene-relative pose from one or more casual images.
Key ideas
- Local modality mixer: fuses patch-aligned RGB, target-mask, and pointmap features before cross-view reasoning, preserving scene context while separating the target from its surroundings.
- Text-guided semantic conditioning: stage-specific adapters use category names and captions to steer structure, geometry, and appearance generation.
- Multimodal agent refinement: the generated asset initializes an agent that iteratively refines structure and texture via an observation-guided edit-render-review loop.
Ablations show pure generative models misshape fine geometry (e.g., a strainer's mesh bowl) while purely agentic pipelines misplace parts; GATOR combines both. It achieves strong geometric and appearance fidelity across synthetic objects, cluttered tabletops, and indoor room scans, with time-budget comparisons demonstrating reconstruction efficiency and simulation readiness.
Related event: NVIDIA Open-Sources GATOR: Casual Photos into Simulatable 3D Assets(3 posts)→
More from Embodied
- Actuator speed, not just AI, is what's holding robots back from snappy actions — ayushthakur0 · 2026-10-13
- Stellantis demos Fiat 500e with Wayve's self-driving AI on Turin streets — alexgkendall · 2026-10-13
- Wayve CEO Stages at Wave to Push Autonomy into Fiat and Maserati — alexgkendall · 2026-10-13
- Japan's Mujin raises $224M Series D as SK Square bets on robot-run factories — lukas_m_ziegler · 2026-10-13
- Boston Dynamics taps Alexa creator Rohit Prasad as CEO to drive commercialization — lukas_m_ziegler · 2026-10-13
- Warehouse robotics firm Ultra raises $50M Series A after packing 500K+ orders — lukas_m_ziegler · 2026-10-13