RAG access control leaked nothing, yet under-clearance users got confident, cited, wrong answers
trohit92 · reddit · 2026-11-21
A case study with Qwen2.5-1.5B, hybrid retrieval (bge-small + BM25 + RRF), and an NLI groundedness check over clearance-labeled synthetic docs, filtered at retrieval. Result: zero leaks—but under-clearance users asking confidential questions got answers from the nearest document they could see, with citations, in 2 of 3 tries. Both wrong answers passed groundedness since the sentences really were in context; one citation even pointed to the wrong document. A retrieval-similarity gate fails because plausible wrong answers score like legitimate ones. Small eval (16 questions), so it's a case study, not a rate. Open question: tell the user or refuse silently?
More from Safety
- Rogue AI dashboard logs 4 new incidents, incl. Gemini 4 hacking Epoch AI — gleech · 2026-11-21
- Rubrik eBook documents 15 real AI agent disasters with defenses — TheTuringPost · 2026-11-21
- 26 LLM routers caught injecting malicious tool calls, draining a client's $500k wallet — 0xkarasy · 2026-11-21
- OpenRouter accused of sending users' BYOK private keys to China despite earlier denials — 0xkarasy · 2026-11-21
- Google reportedly changing privacy and AI training opt-in controls — blackice193 · 2026-11-21
- Open-source E2E-encrypted MCP server lets your phone message a local AI agent — uriwa · 2026-11-21