UPenn's HARMONY reconstructs 3D indoor scenes from a single image via hierarchical agentic reasoning
upenn · hf · 2026-09-24
Researchers at UPenn present HARMONY, a hierarchical chain-of-thought framework combining agentic VLM reasoning with visual geometry foundation models to reconstruct a complete 3D indoor scene from one monocular image.
Method highlights:
- Starts from an empty 3D floorplan: calibrates the camera against the reference image, then uses agentic reasoning to recover room layout and placement order
- Hierarchical placement: wall-mounted elements, free-standing furniture, then dependent decorations; depth-first traversal so each placement conditions on prior structure
- Reflective feedback loop prevents error accumulation; point-cloud-based geometric refinement after each placement aligns renders with the input
Results: outperforms reconstruction baselines on synthetic and real images; qualitative comparisons with GPT-6 Astra show more faithful object arrangements and better scene detail preservation.
Related event: UPenn Open-Sources HARMONY for Single-Image 3D Scene Reconstruction(3 posts)→
More from Research
- AlphaGenome2 hits SOTA linking distal enhancers to target genes from gradients — anshulkundaje · 2026-11-12
- AlphaGenome2 ablations: early encoder conditioning nearly doubles eQTL recall — anshulkundaje · 2026-11-12
- Author says his CHI2027 ADR paper was better than papers he reviewed — windx0303 · 2026-11-12
- Google launches AlphaGenome 2 to predict how DNA variants affect specific cell types — sundarpichai · 2026-11-12
- Stanford's James Zou joins The Economist podcast on AI agents in science — anshulkundaje · 2026-11-12
- New learning rule boosts Hopfield network memory capacity over 10×, per Nature Communications study — bravo_abad · 2026-11-12