EgoTools: 100-Hour Egocentric Dataset Shows Video Models Struggle at Tool Reasoning
ropedia-ai · hf · 2026-10-02
Researchers introduce EgoTools, the first comprehensive suite for egocentric tool-use understanding, with two components:
- EgoTools-Data: 100 hours of tool-centric egocentric recordings with synchronized audio, dense captions, reasoning-heavy narrations, and supplementary 3D information.
- EgoTools-Bench: 1,000 QA pairs across four tracks covering perception, geometry, procedure, and causal reasoning.
Results show current video models struggle to ground tool use in visual evidence: Gemini-3.1-Pro reaches 66.9% overall but only 51.7% on Perception & Grounding. The data also serves as a training resource: full supervised fine-tuning lifts Qwen3-VL-8B-Instruct from 50.0% to 60.9% under strict source-video separation.
Related event: EgoTools: 100-Hour Egocentric Dataset for Tool-Use Reasoning(4 posts)→
More from Embodied
- ETH's Anymal robot completes 10km autonomous alpine hike with VLM-guided ROAM system — ChongZzZhang · 2026-11-19
- iFixit tears down Apple HomeView: lots of glue and 'endless frustration' — Resident-Fall4351 · 2026-11-19
- Removing human video drops robot OOD success from 60% to 7%: generalization comes from human video — gui_penedo · 2026-11-19
- PSYONIC's prosthetic Ability Hand now used by 90+ robotics orgs, raising Series A — DhowCIO · 2026-11-19
- Simulator misspecification breeds overconfidence: sim-to-real study on SPEED+ — Nikolay Dolgov · 2026-11-19
- Doppel world model renders the robot body and beats action-only baselines across 8 embodiments — mangahomanga · 2026-11-19