RAG access control leaked nothing, yet under-clearance users got confident, cited, wrong answers

trohit92 · reddit · 2026-11-21

A case study with Qwen2.5-1.5B, hybrid retrieval (bge-small + BM25 + RRF), and an NLI groundedness check over clearance-labeled synthetic docs, filtered at retrieval. Result: zero leaks—but under-clearance users asking confidential questions got answers from the nearest document they could see, with citations, in 2 of 3 tries. Both wrong answers passed groundedness since the sentences really were in context; one citation even pointed to the wrong document. A retrieval-similarity gate fails because plausible wrong answers score like legitimate ones. Small eval (16 questions), so it's a case study, not a rate. Open question: tell the user or refuse silently?

Original post →

More from Safety

Safety channel →