GPT-6 Astra unlocks agentic real2sim: vision models as tools to turn room video into interactive sims
ericjang11 · x · 2026-09-28
Eric Jang rounds up recent work where GPT-6 Astra has enabled new results in robotics and inverse graphics.
The highlighted example is agentic real2sim: instead of asking an LLM to build a Blender scene from a room video, vision models are handed to GPT-6 as tools — Pi3X for geometry, SAM3 for segmentation, and GVHMR/Kimodo for human motion capture — turning a single room video into a higher-quality interactive simulation.
Jang also invites robotics researchers working on robotic control or agentic real2sim to DM for free access to open-source models (Kimi K3, Qwen 3.8 Flash Next) to benchmark their agentic capabilities against GPT-6.
More from Embodied
- Builder brings a physical Codex Pet to San Francisco for OpenAI Dev Day — paw_lean · 2026-09-28
- Researchers debate GPT-6 Astra: a generalist VLA that could beat custom VLAs — YouJiacheng · 2026-09-28
- Eric Jang: now is the window to test OSS models on robotics before the gap closes — ericjang11 · 2026-09-28
- 10 NeurIPS 2026 robotics papers to watch: VLA inference and long-horizon tasks — shaohua0116 · 2026-09-28
- Claude Opus 5 Shows Creative Tool-Regrasp Skills in Robot Manipulation Tasks — ericjang11 · 2026-09-28
- Eric Jang Offers Free Tokens to Benchmark Open Models vs GPT 6 Astra in Robotics — ericjang11 · 2026-09-28