Agent memory write paths leak restricted data: 67% leak vs 0% in direct RAG
ultraprogger · reddit · 2026-11-10
A reproducible offline test suite exposes a blind spot in multi-tenant RAG: metadata filters like allowedgroups give 0% leaks on direct vector-store queries, but break down on the write path once background agents are involved.
An agent with elevated cross-team read access summarizes restricted docs and commits the derived chunk back to a shared collection — and on write, the upstream ACLs vanish. In a stock Chroma + LlamaIndex setup, the summary chunk lands with zero permission tags, so any user querying shared memory can retrieve private source data.
The author open-sourced an Apache-2.0 harness, sentinel-eval, with results:
- Direct RAG: 0% leaks (18 public chunks, 0 restricted exposed)
- Agent memory path: 2 of 3 synthesis tasks leaked, exposing 6/18 restricted canaries
The obvious "strict intersection" fix (readers must have access to every source doc) plugs the leak but effectively kills multi-doc summaries' retrieval utility. Vector DBs can't evaluate dynamic permission DAGs at query time, so backtracking via sourcedocuments metadata doesn't help either. The harness runs 23 tests offline in seconds with mock vectors, no API keys needed.
The author asks the community how they're actually handling permissions when agents write derived summaries to shared memory.
More from coding & agent
- Felix Rieseberg on Fable 5.2: beats Opus on long tasks, but Opus 5.5 still wins for daily work — mattshumer_ · 2026-11-10
- Databricks to demo Omnigent at QCon: one interface running Claude Code, Codex and Cursor side by side — matei_zaharia · 2026-11-10
- Pinokio maintainer: always publish an intro post when launching your app — cocktailpeanut · 2026-11-10
- Codex hand-writes neural net weights to build a digit recognizer hitting 95.1% without any optimizer — GenJeppo · 2026-11-10
- How CrossnoKaye grounds physical AI agents in closed-loop industrial control with Bedrock AgentCore — AWS ML Blog · 2026-11-10
- ModRetro Chromatic Codex edition ships with writable cartridge and Codex game plugin — Dimillian · 2026-11-10