VideoMemBench: new benchmark exposes major gaps in MLLMs' in-context video memory
Zhengdong Hu · hf · 2026-11-05
VideoMemBench tests MLLMs on long multi-turn dialogues over a single video, where questions depend on earlier turns via Episodic Dependencies (what the model saw) and Answer Dependencies (what it said). Each dependency is verified for necessity and non-leakage, with an LLM planner/generator/judge pipeline plus human review. The benchmark spans 326 videos and 2,476 turns, 1,387 of which depend on earlier context, revealing substantial performance gaps in a capability prior video QA benchmarks never measured.
More from Multimodal
- Atmee v2 live-generates a Taipei alley, pointing toward an agentic world model — flngr · 2026-11-05
- Local demo of Ming-3.0-audio-flash: transcription, sounds and vocal emotion — truecakesnake · 2026-11-05
- New hyper-realistic Glif AI video of a man collecting chestnuts in the rain is nearly indistinguishable from reality — ainsoph00 · 2026-11-05
- CCORE releases final 4B model that generates 3D assets as Three.js scenes with GLB export, weights on Hugging Face — ConfidentDinner6648 · 2026-11-05
- Ant Open-Sources Ming-3.0 Audio Family: TTS with Voice Cloning & Emotion Control, MIT Licensed — Promptmethus · 2026-11-05
- 14-run same-seed test: Regenerate 2K's detail comes from the refiner LoRA, but it changes real people's faces — ReferenceFluffy6159 · 2026-11-05