New RL paper: C-Slerp learns 5-cube stacking from scratch with no rewards or pretraining
ben_eysenbach · x · 2026-10-20
Ben Eysenbach shares a goal-conditioned RL paper (9 months in the making): C-Slerp solves extremely long-horizon tasks like 5-cube stacking from scratch—no prior data, no rewards, no pretrained models. It learns hyperspherical contrastive representations and infers waypoints in closed form. He frames it three ways: hierarchical RL with a closed-form high-level policy; a planning algorithm mapping nonlinear dynamics to a latent space where planning becomes a Brownian Bridge; and representations enabling horizon generalization. He also reframes exploration: representations encode environment structure, unseen goals get mapped into them, and representations update when the goal isn't reachable—explaining how agents reach goals random exploration would never stumble upon.
More from Embodied
- Robotics take: IK guarantees no precision edge, hardware—not the solver—is the bottleneck — ChongZzZhang · 2026-10-20
- MPC vs RL for robot tracking: the precision bottleneck is hardware, not the solver — ChongZzZhang · 2026-10-20
- The most important AI device of the next decade won't be a smartphone — VraserX · 2026-10-20
- NUS MAGIC Lab is hiring a postdoc, 2 PhD students and 3 research assistants for robotics — DJiafei · 2026-10-20
- AlphaNIST two-finger gripper screws nuts multiple ways, "it's just PPO" — _Stocko_ · 2026-10-20
- Boring Company's Prufrock Liner Truck now hauls 20-foot shipping containers through tunnels — elonmusk · 2026-10-20