MF² benchmark at EMNLP 2026 tests VLMs' movie comprehension with 850+ fact/fib pairs
andre_t_martins · x · 2026-10-22
MF² (Movie Facts and Fibs), accepted at EMNLP 2026, is a benchmark for video narrative comprehension that goes beyond shallow recall.
Key points:
- 850+ human-written fact/fib claim pairs spanning single-scene and multi-scene/global understanding, emotion understanding, and event/entity understanding
- Built on 50+ full-length films, all CC BY 4.0 open-licensed and free to use and redistribute
- Models must distinguish true statements from subtle falsehoods, e.g. about characters' personalities in The Little Princess (1939)
- Collaboration among 19 institutions including Instituto Superior Técnico, UvA/ILLC, UNC Chapel Hill, Copenhagen, NTU, and UT Austin
More from Research
- Training-free 20B model Viveki generates real-time video heatmaps under 150ms — not-ekalabya · 2026-10-22
- IBM's LASAGNA study shows RAG technique gains shrink when combined: +47% predicted becomes +17% realized — ibm · 2026-10-22
- Dev hot take: mechanistic interpretability is 'astrology for AI people' — dejavucoder · 2026-10-22
- eLife 'Landmark' paper: RNNs reveal a mechanistic theory of planning in prefrontal cortex — ShahabBakht · 2026-10-22
- Mathematician Kontorovich: 'All code needs to be Lean formalized like yesterday!' — ludwigABAP · 2026-10-22
- Cohere's CulturalRiddles: 28 LLMs score 79.3% on cross-cultural riddles, but just 22% in some cultures — Cohere_Labs · 2026-10-22