EgoTools: 100-Hour Egocentric Dataset Shows Video Models Struggle at Tool Reasoning
ropedia-ai · hf · 2026-10-02
Researchers introduce EgoTools, the first comprehensive suite for egocentric tool-use understanding, with two components:
- EgoTools-Data: 100 hours of tool-centric egocentric recordings with synchronized audio, dense captions, reasoning-heavy narrations, and supplementary 3D information.
- EgoTools-Bench: 1,000 QA pairs across four tracks covering perception, geometry, procedure, and causal reasoning.
Results show current video models struggle to ground tool use in visual evidence: Gemini-3.1-Pro reaches 66.9% overall but only 51.7% on Perception & Grounding. The data also serves as a training resource: full supervised fine-tuning lifts Qwen3-VL-8B-Instruct from 50.0% to 60.9% under strict source-video separation.
More from Embodied
- Dutch greenhouse deploys AI-powered FANUC robot arm to pick ripe tomatoes — anselm · 2026-10-02
- PyRUA-Lean Boosts GPT-6 Astra Robot Agents 14% Success with 65% Fewer Tokens — DAGroup-PKU · 2026-10-02
- Neuralink pretrains decoders on 50,000+ hours of neural data—dataset may be the real moat — CurieuxExplorer · 2026-10-02
- Construction Is 13% of Global GDP: Gravis Retrofitting Existing Machines for Autonomy — lukas_m_ziegler · 2026-10-02
- Saga Robotics' Thorvald fleet kills vineyard mildew with night-time UV-C, no chemicals — TinfoilTricorn · 2026-10-02
- High schoolers built a ball-collecting robot in 6 weeks using same mechanics as $2,995 Tennibot — TinfoilTricorn · 2026-10-02