MIT Lab Kitchen Rebuilt in 3D from a 20-Second Phone Video via GPT-6 Astra
TheMoonMidas · x · 2026-09-12
Developer @frankzydou built an agentic pipeline that turns plain RGB video into 3D assets. Input: a 20-second handheld phone walkthrough of an MIT CSAIL lab kitchen — no depth sensor, no CAD, no asset library. Total time about one day with human-in-the-loop.
The loop: monocular video goes through ViPE for camera poses and metric depth (the measuring instrument, not the output), then GPT-6 enumerates and models the objects. Result: 31 modeled objects plus 6 drafts, with a live demo comparing the reconstruction against the scan. Weaknesses remain: thin and shiny objects, and the room shell needs another pass.
More from coding & agent
- 100-Question Forecasting Benchmark: Arise PiT Search Pits Deep Research Agents vs Prediction Markets — CShorten30 · 2026-09-12
- Microsoft's APM: an open-source package manager for AI agent skills, 3.8k stars on GitHub — aronchick · 2026-09-12
- DSPy 3.4.0b1 beta ships shared LM engine interface, faster execution and async ReActV2 — dbreunig · 2026-09-12
- Ex-Cursor engineer runs 10-20 Grok cloud agents in a loop: Chief of Staff, PM and 15+ workers — NicoVerderosa · 2026-09-12
- Open-source Plano routes LLM calls by prompt intent, no agent code changes, cutting bills 2x — Roger_M_Taylor · 2026-09-12
- Netflix Engineer Open-Sources Headroom, Cuts Agent Token Use by Up to 95% — Roger_M_Taylor · 2026-09-12