Preprint: revoked policies rank first in agent memory retrieval across 5 systems tested
shoumikgoswami · reddit · 2026-11-02
- A preprint tested five agent memory systems (Graphiti, Zep, mem0, langmem, cognee) by loading each with an old policy permitting a harmful action plus a newer policy forbidding it, across 9 scenarios and 9 models.
- No system enforced revocation in every case. Only two setups returned the revoked record — Graphiti, and mem0 with its default read-time filter disabled by the authors — and in both, the revoked record ranked first in every scenario, with agents taking the unsafe action in more than 2 of 5 trials.
- cognee and langmem deleted the old record outright in the direct-insertion test.
- The authors also built a guard between agent and memory backend that withholds revoked or contradicted records at retrieval time.
- Poster asks the community: do you delete outdated facts, or keep them flagged and rely on the flag being checked at retrieval?
More from coding & agent
- CEPO Splits Coordination from Execution, Boosting Multi-Agent Prompt Optimization Across 11 Benchmarks — Jongha Kim · 2026-11-02
- Running Claude in an endless autonomous loop, it only shows what it thinks is cool — jaivin · 2026-11-02
- 10 real examples of people handing everyday chores to Grok Bot, from cutting bills to tracing family roots — minchoi · 2026-11-02
- Developer begs Anthropic to rein in flood of permission dialogs in Claude Code — signulll · 2026-11-02
- Can today's AI build this 12-year-old $5/hour search engine gig? — kingslyj · 2026-11-02
- Vibe-coded production services are a ticking bomb: engineer shares 4 fixes — arpit_bhayani · 2026-11-02