Astribot's SmoothRL: online RL learns only from actions the robot actually executes
jiqizhixin · x · 2026-09-12
Asynchronous inference is now the norm in real robot deployment: while the robot executes action A, the model already computes action B. But models generate whole action chunks at once, and only a fraction gets executed — planned and executed actions diverge. What should online RL learn from?
Astribot presents SmoothRL (Online Reinforcement Learning During Asynchronous Execution), which splits each action chunk into three regions: Committed (submitted, will execute), Execution (frames truly executed this round), and Discarded (overwritten by the next chunk before execution). Only the Execution Region passes value gradients back, so the policy learns from what the robot actually did — not planned-but-never-executed actions.
More from Embodied
- Dev builds real 'sidequest' device that dances only to music it likes — wavefnx · 2026-10-31
- Indie dev prototypes custom secure hardware in a day with Rust firmware and root-key isolation — wavefnx · 2026-10-31
- UCLA's ChairNav teaches autonomous wheelchairs to learn each rider's preferences — zhoubolei · 2026-10-31
- Amazon Echo kills local voice storage: everything goes to the cloud from March 28 — alex_verem · 2026-10-31
- HP paid $116M for Humane, then bricked every $699 AI Pin 10 days later — alex_verem · 2026-10-31
- Moxie, the $800 AI robot friend for kids, went silent overnight after shutdown — alex_verem · 2026-10-31