FondueQA benchmarks how LLMs compensate for missing retrieval evidence from memory
_reachsumit · x · 2026-11-10
FondueQA is a multi-hop QA benchmark where every required fact is uniquely localized to one evidence document, enabling controlled document-level ablations to study parametric compensation without benchmark contamination.
Findings: compensation strongly depends on entity popularity and the structural role of missing evidence; models compensate better for popular entities; bridging evidence can matter more than the answer-bearing document itself; and models may recall a fact in isolation yet fail to use it under partial evidence, especially with plausible distractors in context.
More from Research
- Claude Fable 5.2 tops VIALS-AA science visual reasoning benchmark at 67% with heavy tool use — ArtificialAnlys · 2026-11-10
- WVU researcher presents 10 years of tractometry work on Alzheimer's white matter vulnerability — PTenigma · 2026-11-10
- New cryo-EM structure: pSTAT1 bound to importin alpha 5 (PDB 11GJ) — chaitjo · 2026-11-10
- ARLoop: fully-open 1.2B looped transformer beats Huginn-3.5B and Qwen3.5-2B on all 8 benchmarks — hhsun1 · 2026-11-10
- RAGCap-Bench: A Fine-Grained Benchmark for Intermediate Capabilities in Agentic RAG — kalyan_kpl · 2026-11-10
- Networks Aren't Scale Invariant, Yet Countless Methods Assume It — Why? — Jianlin_S · 2026-11-10