STEAM: A New Robot Offline RL Framework
青稞AI · wechat · 2026-07-20
This WeChat article introduces **STEAM**, a frame-wise advantage estimation framework for real-world robot learning proposed by Prof. Yu Chao's team at Tsinghua and Zhengxing Innovation, **requiring no manual frame-by-frame annotation**. ## Core Idea - To address the common mixture of pauses, corrections, failures, and suboptimal actions in real robot data, STEAM uses **normalized temporal offsets** between frame pairs in expert trajectories as a self-supervised signal. - By training multiple temporal-offset predictors, the results are converted into advantage scores measuring local task progress. - To handle mixed-quality rollouts, the method adopts **the minimum advantage value from an ensemble model** as a conservative score, more robustly identifying effective progression, failures, and recoveries. ## Experimental Results - In real robot tasks such as dual-arm towel folding, chip checkout, cola restocking, and single-arm grasping and placing, STEAM effectively identifies low-quality segments. - Combined with CFGRL, performance improvements reach **59% / 54.3% / 23% / 16.2%** over baselines. ## Additional Information - The article also previews a live stream on July 21, where STEAM first author Liu Zhihao will explain the method, compare existing approaches, discuss the framework, and present real robot experiments.
More from Embodied
- Generative Bionics turns its Gene.01 humanoid from CES concept into a walking platform — CyberRobooo · 2026-07-21
- Auki pitches physical AI infrastructure for stores, warehouses and supply chains — broodsugar · 2026-07-21
- Safe RL paper on directional constraints accepted to IROS 2026 — Jan_R_Peters · 2026-07-21
- Sunday Robotics says one example can teach its home robot a new behavior — saranormous · 2026-07-21
- San Francisco event seeks sponsors for the first 6-foot humanoid robot fight — cixliv · 2026-07-21
- Tiny memristor chip brings brain-model inference under 10 ms — striketheviol · 2026-07-21