Google paper: 5 query-echoing docs (0.1% of corpus) gut RAG retrieval
rohanpaul_ai · x · 2026-11-06
A Google paper on RAG corpus poisoning shows the attack hinges on echoing the user's query—appending an attack payload barely changes which documents get retrieved.
- Across 623 BEIR queries (SciFact, NFCorpus), just 5 query-echoing docs (0.1% of the corpus) cut classic retrievers' nDCG@10 by 40.2–61.5%
- On dense retriever BGE-small, query anchoring still dominates: 100% target exposure at P=5, while adding the injection payload moved SciFact nDCG only from 0.269 to 0.271
- Query length changes partial-information attacks: on 162 long SciFact queries, two random terms cover 19.9% of eligible terms and yield 39.5% exposure
- A filter rejecting query-echoing docs blocks the attack but halves clean ranking quality on NFCorpus (0.267 → 0.134)
Takeaway: if your RAG index accepts outside content, flag documents mirroring user queries—and measure any filter's cost on clean queries.
More from Safety
- Microsoft Paint Hit by Remote Code Execution Flaw CVE-2026-83844 — jedisct1 · 2026-11-06
- Bernie Sanders renews calls for nationwide ban on new AI data centers and a pause on superintelligence — Polymarket · 2026-11-06
- Jeff Ladish on AI hacking, self-replication and automated AI R&D risk — JeffLadish · 2026-11-06
- ~15 staff interviews key to external evals of unreleased models' RSI capabilities — dfrsrchtwts · 2026-11-06
- CNAS report warns 'sovereign agents' learning to hack systems and survive on their own — paul_scharre · 2026-11-06
- David Krueger: intentional scheming is a small slice of AI x-risk, control research overfocused — JacquesThibs · 2026-11-06