Multi-Hop RAG Failures Come From Search Drift, Not Retrieval Limits
_reachsumit · x · 2026-10-16
The arXiv paper "Beyond Retrieval Failure" examines why LLMs miss evidence in multi-hop RAG. Standard retrieval evaluations conflate two causes: a passage being inaccessible to the retriever versus the agent failing to issue a query that would retrieve it.
- Introduces ExecutionRAG, a training-free requirement register that records open sub-questions and their supporting evidence. It changes search guidance but not evidence access, so every gold passage it recovers certifies the passage was reachable by the original retriever.
- On 1,000 MuSiQue questions, wrong answers with unretrieved required evidence drop from 382 to 234 with all-MiniLM-L6-v2 and from 383 to 274 with Qwen3-Embedding-0.6B — 69%→44% and 66%→48% of each system's wrong answers.
- Crucially, evidence recovery does not proportionally reduce total error: premature resolutions and abstentions rise as missing-evidence failures fall, shifting the remaining bottleneck away from search guidance.
More from coding & agent
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16
- Builder's one rule for trusted desktop agents: read freely, but sending, deleting, buying always ask first — No_Fan5310 · 2026-10-16
- GPT-built unifi-simulator makes MikroTik switches appear inside UniFi Network — Pytlicek · 2026-10-16
- Stanford drops 45-min SpaceX lecture on how Grok Bot's always-on agent harness works — Hesamation · 2026-10-16
- Opus 5.5 + Three.js demo stuns with water physics, fish and blossoms — nikola_mr64990 · 2026-10-16
- Grok-built linear regression bot claims $508K PnL in 125 days on Polymarket — Roger_M_Taylor · 2026-10-16