StreamGaze, first benchmark for gaze-guided temporal reasoning in streaming video, accepted at NeurIPS
mohitban47 · x · 2026-10-03
- StreamGaze, developed with Adobe Research, has been accepted to the NeurIPS 2026 E&D track (scores 5/5/4). It is the first benchmark evaluating whether MLLMs can use human gaze signals for temporal reasoning over streaming egocentric videos.
- The benchmark spans past, present, and proactive reasoning across 10 tasks, from gaze-sequence matching to alerting when objects enter the field of view.
- A data construction pipeline aligns egocentric videos with raw gaze trajectories via fixation extraction, region-specific visual prompting, and scanpath construction to produce spatio-temporally grounded QA pairs.
More from Multimodal
- AI Animated Short 'The Solar Ascendant' Shows Pixar-Grade Flaming Monkey Hero — Solaris-Frame · 2026-11-19
- AI video brings Mona Lisa, Van Gogh and Dalí together across centuries — sebpaquet · 2026-11-19
- Sound Alike Matches Any Video Voice to Its Musical Twin and Builds a Music Video — VidaB98 · 2026-11-19
- Fully Seedance 2.5-generated short shows AI video getting comedically coherent — anthara_ai · 2026-11-19
- Kling Image 4.0 debuts at #7 on Artificial Analysis, only model with native 8K output — anthara_ai · 2026-11-19
- Creator explains his slopcore video workflow built on a Claude plugin — AIandDesign · 2026-11-19