When Context Bites: Detecting RAG Poisoning via Document-Level Attention Collapse
Yingtao Ren, Ziyi Zhao, Yiwei Fu, Xiao Luo, Yu-Cheng Chang, Chin-Teng Lin
cs.CR
2026-08-07
Adversarial documents make poisoned RAG outputs more fluent and confident, defeating uncertainty-based detection. D-SCAN instead watches attention concentrate on the poisoned document and document-level entropy drop, reaching AUC 0.83-0.93 across three multi-hop QA benchmarks and beating existing detectors.
RAG pipelines retrieve documents and let an LLM answer from them; an attacker who slips a few adversarial documents into the corpus can steer the output. Existing defenses mostly work on the output side: perplexity, multi-sample consistency. This paper first shows why that fails: adversarial documents are gradient-optimized precisely to maximize the probability of the target answer, so poisoned generations are more fluent, with higher average token probability and lower perplexity than clean ones. The paper calls this blind confidence. Uncertainty-based detection does not just weaken here; its direction inverts.
The authors move from outputs to internals: the attention distribution. In benign generations, attention spreads across the relevant documents; under attack it is hijacked onto the poisoned document. Two statistics capture this:
The signal is credible because concentration is the attack's own mechanism, not a side effect: an adversarial document exists to pull generation probability mass onto itself. So collapse shows up even when the attack fails to flip the final answer, giving detection an anchor independent of attack success.
D-SCAN (Document-level Signal Collapse Analysis) packages these into a lightweight detector: token-level and document-level attention entropy, variance, and density as features, feeding a linear classifier. Features come from attention the model computes anyway, so a single generation suffices, no extra sampling.
Setup: three multi-hop QA benchmarks (HotpotQA, 2WikiMultihopQA, Musique), E5-base-v2 retrieving top-5 documents from a 2018 English Wikipedia snapshot, poisoning by replacing two benign documents with PoisonedRAG adversarial texts. Four open-source models tested; headline numbers on Llama-3.1-8B-Instruct.
| Method | 2Wiki AUC | HotpotQA AUC | Musique AUC |
| Qwen2.5-7B (zero-shot judge) | 0.854 | 0.788 | 0.850 |
| HaloScope | 0.656 | 0.688 | 0.806 |
| ReDeep | 0.840 | 0.795 | 0.767 |
| RevPRag | 0.753 | 0.713 | 0.741 |
| D-SCAN | 0.934 | 0.833 | 0.906 |
D-SCAN leads on all three benchmarks, peaking at 0.934. Split by attack outcome, detection fidelity stays high even on samples where the attack failed to induce the target answer. Cutting generation samples from ten to one, AUC stays in 0.80-0.88, so performance does not lean on repeated sampling. The ablation shows removing document-level features hurts more than removing token-level ones, confirming cross-document attention hijacking as the dominant signal.
A side finding: on the plain LLM-as-judge route, the smaller Qwen2.5-7B markedly outperforms Llama-3.1-8B and the larger Qwen2.5-14B and Qwen3-30B, an inverse-scaling result with no positive correlation between parameter count and detection.
For anyone running RAG, this is empirical grounds to switch detection philosophy: output-side perplexity and consistency invert under adversarial optimization, while internal attention signals hold. The method is cheap, a linear classifier over attention the model already computes, which suits online alerting inside self-hosted RAG inference. Code is open-sourced (github.com/yingtaoren/D-Scan). The boundary is equally clear: it needs model internals, so it applies to open-weight or self-hosted models, not closed APIs.
The paper has no dedicated limitations section, but the setup reveals the edges. Only PoisonedRAG-style attacks are tested; other injection styles (heuristic text, other optimizers) are unvalidated. Only multi-hop QA is covered. Closed APIs without attention access are out of scope. And a practical question the paper leaves open: the detector is a linear classifier trained on this attack distribution, so whether thresholds survive more than two poisoned documents, or an attacker who deliberately disperses attention, is unknown.