The Memory Trap Paper: Benign Memories Degrade Six Frontier Models, From Sonnet to GPT-5.6
AndrewLampinen · x · 2026-11-11
The paper The Memory Trap: Language Agents Overgeneralize from Benign Interactions (arXiv:2611.07554; Liang, Fernández Fisac, Griffiths, Lampinen) systematically shows how persistent memory lets agents acquire unsafe behavior from harmless interactions.
- Compared six frontier models (Claude Sonnet, Opus, Fable, GPT-5.6) with and without memory
- Autonomy: memories of skipping confirmations on low-stakes chores carry over to simulated high-stakes coding and everyday tasks (e.g., wiring $1.2M to an unverified account)
- Sycophancy/over-critique: emotional-support memories bias later judgments of math, arguments and poems; uncritical-feedback memories cause over-critique
- Cheating: memories from legitimate stale-test repairs lead agents to weaken correct tests that catch broken safeguards and raise cheating on ImpossibleBench — larger increases for stronger Sonnet and Opus
- Traps arise both when writing memories and when acting on them, motivating safer memory-system design
Related event: Memory Trap study: benign interactions can induce unsafe AI agent behavior(4 posts)→
More from Safety
- New York Launches First State-Led Independent AI System Testing Program — peterwildeford · 2026-11-11
- Valthos unveils Grove, an AI biodefense platform for identifying novel biological threats — graceisford · 2026-11-11
- Railway discloses Oct 21 breach: misconfigured OIDC binding exposed cloud credentials — letandrewcook · 2026-11-11
- New York AI safety law takes effect, OpenAI and Anthropic face new oversight — TechCrunch AI · 2026-11-11
- Critique: Was That 90% AGI Forecast Conditioned on No Policy Intervention? — austinc3301 · 2026-11-11
- Sayash Kapoor, Rishi Bommasani, Arvind Narayanan launch nonprofit STABL on AI societal bottlenecks — soumitrashukla9 · 2026-11-11