Robo-ValueRL rates actions by estimated task progress to learn from messy robot data
burkov · x · 2026-11-18
Robo-ValueRL tackles a data-quality problem in robots trained from demonstrations and real-world experience: the data mixes successful actions with hesitation, mistakes, and failed attempts, and treating every action equally limits offline learning and destabilizes online updates.
Key points:
- Estimates task progress from current and recent visual observations
- Uses changes in those progress estimates to weight helpful vs. unhelpful actions across offline and online stages
- Validated on precise chip insertion plus block disassembly and sorting
- Tested with 240 hours of offline robotic data and 3,000+ online rollout trajectories
The framework also measures value-estimate uncertainty (text truncated in source), aiming to make better use of mixed-quality robot experience.
More from Embodied
- "No sim2real gap if you only deploy in sim" — who is building this? — rahulshivchand · 2026-11-18
- Maker adds a new feature to the googly-eye clock and everyone wants one — alexcovo_eth · 2026-11-18
- Figure CEO Bret Adcock: humanoid race has four stages, Figure is now in stage three with Helix 3 as 'baby AGI' — PeterDiamandis · 2026-11-18
- Argon open-sources Alloy, a multi-material physics engine for robot learning on one GPU — ycombinator · 2026-11-18
- California DMV Orders Waymo Off State Freeways After Traffic Incidents — Polymarket · 2026-11-18
- ImpedanceMimic learns per-joint stiffness and damping for force-accurate humanoid motion imitation — carlosdponx · 2026-11-18