Robots get their GPT-3 moment: In-Context Learning lets them learn new tasks from one demo
量子位 · wechat · 2026-09-06
A deep-dive on In-Context Learning spreading to embodied AI: Skild AI's S1 watches one task demo and completes new 10-minute, multi-step tasks without fine-tuning, while Generalist AI's GEN-1.5 does one-shot learning in seconds. Video context reportedly beats language-only prompts by 7x on unseen tasks.
The piece traces the path from GPT-3's ICL to RoboTTT (Fei-Fei Li, Jim Fan, Yuke Zhu) bringing context scaling to robot policies, and interviews Gao Yuxiang, founder of Chinese startup COCO Matrix. He argues that scaling data alone hasn't delivered generalization; the hard parts are hierarchical visual understanding, missing multimodal data (touch/proprioception), and long-horizon memory via KV cache compression. The company bets on "strong understanding, light generation": task-conditioned visual representations that let a 60M action head beat a 1.1B baseline. Next challenges include defining human-teaching interaction datasets.
More from Embodied
- HKU's TILDE Distills Embodied Foundation Models into Robot Policies — hkuhk · 2026-10-23
- 10 of 12 Beta Robots Worked Without Fine-Tuning — Do We Still Need It? — micoolcho · 2026-10-23
- EmbodiedEvo lets robot agents self-train, banking reusable Skills and Lessons across tasks — jiqizhixin · 2026-10-23
- KAIST's FRED lifts robot policy success from 19.4% to 50% via self-distilled future video representations — kaist-ai · 2026-10-23
- GALAX Launches World's First RTX 5080 With Dual 12V-2x6 Connectors at $3000 — chemist_slime · 2026-10-23
- Figure prints 81 minutes of robot data per second, spent ~$8.1M on data last week — joecole · 2026-10-23