Cheap failures work: human videos and real2sim rival kinesthetic teaching for robot reward models
mangahomanga · x · 2026-11-06
The author analyzes how data sources of different collection costs — kinesthetic teaching, teleoperation, policy rollouts, human videos, and real2sim — affect adapting reward models for manipulation.
Across three tasks, all four sources improve average success over the base reward model, with low-effort human videos and real2sim delivering gains approaching high-effort kinesthetic teaching and policy rollouts — a strong case for cheap failure data in real-world RL.
Related event: JHU Study Compares Four Failure Data Sources for Robot Reward Models(7 posts)→
More from Embodied
- Policy trained entirely in imagination matches PPO with 7.9× fewer simulator transitions — EpisodeYang · 2026-11-06
- Apple's mystery robot reportedly uses a Mac Mini as its planner brain — micoolcho · 2026-11-06
- Microdock v2 lets robot sit while docked, cutting servo power to zero — natesiggard · 2026-11-06
- Building an Asimov Robot in 22 Hours — and It Worked Out of the Box — whurley · 2026-11-06
- NEO-VLA drops the vision encoder: native VLA pretraining pays off on RoboTwin — liuziwei7 · 2026-11-06
- Gold Tesla Cybercab and Optimus shown at UP Summit — DMaguireARK · 2026-11-06