GAM: training robot policies from a 3D detector beats larger VLAs, with 1M-image WildDet3D dataset
DJiafei · x · 2026-09-23
Researchers from Allen AI and UW introduce the Grounded Action Model (GAM), challenging the assumption that language/video models are the right foundation for robotics. Core thesis: ground first, then learn to act — train a policy directly on a 3D understanding model rather than a VLM.
Key points:
- The team argues GPT6-Astra shows scaling spatial/3D understanding matters more than scaling action for manipulation
- GAM uses WildDet3D, a small promptable 3D box detector, as its base — and outperforms most larger VLAs and world action models despite its minimalist design
- WildDet3D is a unified geometry-aware monocular 3D detector natively accepting text, point, and box prompts, with optional depth cues at inference
- Its WildDet3D-Data is the largest open 3D detection dataset to date: 1M+ images across 13.5K categories, built by generating candidate boxes from 2D annotations with human verification
- WildDet3D sets new SOTA on multiple benchmarks, scoring 22.6/24.8 AP3D-dist on the new WildDet3D-Bench in open-world settings
Paper, code, models, and dataset are all released, plus an iPhone demo app.
Related event: GAM: Grounded Action Model Proposes 3D Grounding as Robot Foundation Base(6 posts)→
More from Embodied
- Insider pushes back on rumor that Figure's Hugo robot is built on a Unitree G1 — carlosdponx · 2026-10-23
- Sergey Levine: robotics is repeating LLMs' shift from fine-tuning to plain-language prompting — adityaag · 2026-10-23
- Embodied First Exam benchmarks coding agents on 1,090 tasks across 44 robots — ManlingLi_ · 2026-10-23
- Delivery robot spotted cruising city sidewalks on its own — HomeNowWTF · 2026-10-23
- Dev builds ESP32 desk display showing Claude Code and Codex agent status via local Go helper — tonylook · 2026-10-23
- Dream Labs emerges from stealth with Dream-1 robot foundation model at $3/hour — scott_e_reed · 2026-10-23