AI Agents Generate Robot Training Scenarios
MIT News AI · rss · 2026-07-14
MIT CSAIL and Toyota Research Institute introduced SceneSmith: a framework where multiple AI agents collaborate to generate 3D virtual scenes for robot training, aiming to alleviate the shortage of robotic data.
Core Concept
- Uses a division of labor among three agents: design, review, and coordination, to iteratively build indoor scenes.
- Scenes are richer than previous methods, holding up to 6 times more objects per scene.
- The generated environments can be directly integrated into physics simulators for robot task testing and policy evaluation.
Experimental Results
- Researchers generated 1300+ scenes and tested robotic action policies within them.
- Human and model judgments on "policy failure" showed an agreement rate of over 99%.
- Pre-trained robot policies were also placed into these scenes for validation, proving the environments work not just visually, but for physical interactions.
Costs and Limitations
- Generating a single scene can take hours, making speed the main current trade-off.
- The team hopes to expand support to more deformable objects in the future.
More from Embodied
- Shanghai AI Lab-led team releases 100-hour egocentric wearable dataset for humanoid training — ChuckDBrooks · 2026-10-26
- RealSense D585 Pro AI-Powered 3D Stereo Depth Camera Unboxed for Physical AI — chrismatthieu · 2026-10-26
- Physical world agent swarm demo shared on X — pduan · 2026-10-26
- Looking for a wearable mic with best battery life that feeds recordings to personal agents via MCP — altryne · 2026-10-26
- 'Lamp Swarm': a desk of lamp-shaped robots shows what physical AI work looks like — dee_hw · 2026-10-26
- Tencent's HiSTEVE 1.0: hierarchical VLA agent plays Minecraft with native keyboard and mouse, hits 85% SOTA — _akhaliq · 2026-10-26