EgoTSR Tackles VLM Temporal Bias in Robot Task Progress
机器之心 · wechat · 2026-07-04
A joint team from Zhejiang University, Tianjin University, Qingdao University, Shanghai Jiao Tong University, and the National University of Singapore introduced EgoTSR to address the "temporal bias" in Vision-Language Models (VLMs) during robot task reasoning. Because most robot videos are recorded chronologically, models learn a shortcut assuming that later frames are closer to task completion, failing to recognize real-world scenarios like grasping failures or state regressions.
Approaching from a first-person robot perspective, the research enables VLMs to assess task states and scale to long-horizon planning. The team built the EgoTSR-Data dataset containing 46 million samples and designed a three-stage curriculum learning process: establishing explicit reasoning with 15 million CoT samples, internalizing rapid judgments with 16 million Tag samples, and extending to long-horizon tasks with 15 million LongTag samples. They also introduced a SubtaskPlanner to break down high-level tasks into ordered atomic subtasks, allowing the model to track progress along the task chain.
The researchers exposed this sequence dependency using a simple method: inputting the same pair of images in both forward and reverse order. Experiments revealed severe issues—in some long-task evaluations, InternVL-8B achieved nearly 99% accuracy with forward input but plummeted to about 2% when the image order was swapped, proving the model guesses based on image position rather than genuinely analyzing object states. Code is open source.
More from Embodied
- Teachers decry plan to put a humanoid robot in a New York high school — nordicinst · 2026-07-27
- NUS builds a soft force sensor that drives actuators without electronics or power — CurieuxExplorer · 2026-07-27
- Chelsea Finn says robot RL is bottlenecked by physical rollout cost, not algorithms — ycombinator · 2026-07-27
- Robot goes to the fridge and fetches a beer — Darpinian · 2026-07-27
- Researchers show digital circuits can be replicated with knitted fabric — mtizard · 2026-07-27
- Local Qwen models power a robot that tests 78 smartphones’ battery life — gappyvalley · 2026-07-27