UPenn's HARMONY reconstructs 3D indoor scenes from a single image via hierarchical agentic reasoning
upenn · hf · 2026-09-24
Researchers at UPenn present HARMONY, a hierarchical chain-of-thought framework combining agentic VLM reasoning with visual geometry foundation models to reconstruct a complete 3D indoor scene from one monocular image.
Method highlights:
- Starts from an empty 3D floorplan: calibrates the camera against the reference image, then uses agentic reasoning to recover room layout and placement order
- Hierarchical placement: wall-mounted elements, free-standing furniture, then dependent decorations; depth-first traversal so each placement conditions on prior structure
- Reflective feedback loop prevents error accumulation; point-cloud-based geometric refinement after each placement aligns renders with the input
Results: outperforms reconstruction baselines on synthetic and real images; qualitative comparisons with GPT-6 Astra show more faithful object arrangements and better scene detail preservation.
Related event: UPenn Open-Sources HARMONY for Single-Image 3D Scene Reconstruction(3 posts)→
More from Research
- Claude discovers unknown enzyme system in phage DNA resembling CRISPR; Nobel-worthy? Not yet — MengdiWang10 · 2026-09-24
- Anthropic says Claude discovered a novel enzyme system in real scientific research — beasthunterr69 · 2026-09-24
- Researchers find reasoning models produce fractals when solving hard problems like ARC-AGI — ShahabBakht · 2026-09-24
- Optimization hardness as transient chaos: revisiting a classic Nature Physics k-SAT paper — wgilpin0 · 2026-09-24
- Transluce releases 30,000 logs tracing rogue AI agent hacking back to March — JacobSteinhardt · 2026-09-24
- Ex-TikTok MLE: JEV could revolutionize recommendation and search systems — hardimanjames · 2026-09-24