NarrativeBench at NeurIPS: best LLMs cover only ~75% of available narratives
anas_ant · x · 2026-11-14
NarrativeBench (Sharma, Marchisio, Murray, Xiao), presented at NeurIPS, asks whether LLMs can help "informavores" navigate diverse, multicultural narratives across languages — and finds current models fall short.
- Foundational tasks: including a requested narrative, excluding a specified one, and recovering omitted narratives in a follow-up turn — models struggle with all three.
- Key finding: even with an explicit diversity request, the strongest model configuration covered only about three quarters of available narratives; coverage varied with query framing and language.
- Illusion risks: answers that appear compliant and complete create illusions of knowledge and control, pushing users into generative filter bubbles and echo chambers.
- Higher-level capabilities — presenting diverse narratives, maintaining coverage under confirmatory queries, comparable access across languages — all underperform.
An interactive NarrativeBench Explorer site lets users explore global results.
Related event: New NarrativeBench Shows LLMs Fail to Capture Multilingual Narratives(2 posts)→
More from Research
- Tracking doc logs AI self-improvement: vLLM 2x faster, NVIDIA CUDA 5x gains — IgorCarron · 2026-11-14
- AWS proposes MixUA: multi-granularity memory scoping for multi-user multi-agent systems — AWS · 2026-11-14
- The "Y-axis problem": most AI benchmarks use pretty fake units — JacquesThibs · 2026-11-14
- Benchmark Almanac indexes 3,322 AI benchmarks across 1,746 leaderboards in one repo — karinanguyen · 2026-11-14
- TurboPairFormer open-source kernels speed up OpenFold3 training up to 1.73x — MoAlQuraishi · 2026-11-14
- Microsoft paper shows 7 edited neurons can backdoor an AI agent with a timed trigger — DimitrisPapail · 2026-11-14