Three Years On: GPT-4's Eureka Taught a Robot Hand to Spin Pens in Isaac Gym
JasonMa2020 · x · 2026-09-17
Looking back three years, Jason Ma recalls how GPT-4 was left 'cooking' in Isaac Gym and learned pen-spinning tricks at a superhuman level — arguably, by today's terminology, the first RSI moment in robotics. Eureka, announced by Jim Fan, is an open-ended agent that designs reward functions for robot dexterity, described as 'Voyager in the space of a physics simulator API.'
Technical highlights:
- Hybrid-gradient architecture: an outer loop runs inference-only GPT-4 to iteratively refine reward functions (gradient-free), while an inner loop trains a robot controller via reinforcement learning (gradient-based)
- Bridges high-level reasoning (coding) with low-level motor control
- Scaled up thanks to GPU-accelerated Isaac Gym simulation
The authors argue Eureka was ahead of its time, a prescient example of the agent-plus-robot-learning convergence now heating up.
More from Embodied
- Robotics team turns an actuator race condition bug into the feature they needed — eigenron · 2026-09-17
- ArmSoM Sige 7 unboxed: an RK3588 board for edge AI and robotics — chrismatthieu · 2026-09-17
- Closed-room Hong Kong robotics workshop tackles foundation models, data engines, deployment — paigeinsf · 2026-09-17
- Innate OS open-sourced: an agentic OS for general-purpose robots under $1k — ycombinator · 2026-09-17
- Wayve appoints Elisa de Martel as CFO to scale AI driver to millions of vehicles — alexgkendall · 2026-09-17
- Tutor Robotics unveils Sonny, a mobile VLA-powered robot that can push a whole cart — chris_j_paxton · 2026-09-17