Qualcomm's IVD benchmark shows VLMs lag far behind humans at real-time real-world Q&A
rishit_dagli · x · 2026-10-02
A Qualcomm team introduces the Interactive Video Dataset (IVD), a benchmark testing whether VLMs with a camera and microphone can answer user questions about live scenes in real time — a prerequisite for real-world AI assistants and humanoid robots. Existing models fall far behind human performance, but fine-tuning on this data significantly narrows the gap, and the paper shares data and training techniques for such settings.
Related event: Qualcomm's IVD Benchmark Shows VLMs Lag Far Behind Humans in Real-Time Q&A(2 posts)→
More from Embodied
- Open-source Harness fork moves coding agents out of the app into orca — dee_hw · 2026-10-02
- YC F26 startup Preload captures synchronized video, EMG and tactile data for robots — ycombinator · 2026-10-02
- Peking University's DexPolicy lifts dexterous manipulation success to 85% via annealed exploration — PekingUniversity · 2026-10-02
- Robotics researcher: domain randomization trades away precision, classical control has better guarantees — KyleMorgenstein · 2026-10-02
- 'A robot without people to design and fix it is inventory, not revenue' — MatthewChang · 2026-10-02
- Galaxy General's ET1 humanoid starts at 79,000 yuan, learns dance moves by watching — 量子位 · 2026-10-02