EgoTSR Tackles VLM Temporal Bias in Robot Task Progress
机器之心 · wechat · 2026-07-04
A joint team from Zhejiang University, Tianjin University, Qingdao University, Shanghai Jiao Tong University, and the National University of Singapore introduced EgoTSR to address the "temporal bias" in Vision-Language Models (VLMs) during robot task reasoning. Because most robot videos are recorded chronologically, models learn a shortcut assuming that later frames are closer to task completion, failing to recognize real-world scenarios like grasping failures or state regressions.
Approaching from a first-person robot perspective, the research enables VLMs to assess task states and scale to long-horizon planning. The team built the EgoTSR-Data dataset containing 46 million samples and designed a three-stage curriculum learning process: establishing explicit reasoning with 15 million CoT samples, internalizing rapid judgments with 16 million Tag samples, and extending to long-horizon tasks with 15 million LongTag samples. They also introduced a SubtaskPlanner to break down high-level tasks into ordered atomic subtasks, allowing the model to track progress along the task chain.
The researchers exposed this sequence dependency using a simple method: inputting the same pair of images in both forward and reverse order. Experiments revealed severe issues—in some long-task evaluations, InternVL-8B achieved nearly 99% accuracy with forward input but plummeted to about 2% when the image order was swapped, proving the model guesses based on image position rather than genuinely analyzing object states. Code is open source.
More from Embodied
- Johns Hopkins Launches Full-Stack Hands-on Robot Learning Class with SO-101 Arm Kits — _krishna_murthy · 2026-09-11
- SyncWorld: In-Context Robot World Model Simulates Unseen Views and Embodiments Zero-Shot — ChongZzZhang · 2026-09-11
- A 3D Pose Dataset for Dogs Released — ducha_aiki · 2026-09-11
- Swaayatt demos autonomous driving at 52 km/h on mountain roads, self-recovers after skid — sanjeevs_iitr · 2026-09-11
- AUAR's MicroFactory brings a deployable robotic wood-panel factory to the construction site — lukas_m_ziegler · 2026-09-11
- Musk: Cybercab certified at 165 Wh/mi, the most efficient production EV ever — elonmusk · 2026-09-11