More failure data isn't always better: JHU finds reward quality plateaus then degrades
mangahomanga · x · 2026-11-06
From the JHU robot RL paper thread (4/n): more failures do not always make better rewards.
Scaling analysis shows adding failure data improves offline reward quality initially, then plateaus or even hurts it — cheap sources (human videos, real-to-sim) help at low data volumes but can degrade reward modeling beyond that.
Across three tasks, all four failure sources improved average success over the base reward model, with human videos and real2sim delivering strong gains approaching kinesthetic teaching and policy rollouts.
Related event: JHU Study Compares Four Failure Data Sources for Robot Reward Models(7 posts)→
More from Embodied
- Policy trained entirely in imagination matches PPO with 7.9× fewer simulator transitions — EpisodeYang · 2026-11-06
- Apple's mystery robot reportedly uses a Mac Mini as its planner brain — micoolcho · 2026-11-06
- Microdock v2 lets robot sit while docked, cutting servo power to zero — natesiggard · 2026-11-06
- Building an Asimov Robot in 22 Hours — and It Worked Out of the Box — whurley · 2026-11-06
- NEO-VLA drops the vision encoder: native VLA pretraining pays off on RoboTwin — liuziwei7 · 2026-11-06
- Gold Tesla Cybercab and Optimus shown at UP Summit — DMaguireARK · 2026-11-06