Mnemos vs Claude Code memory test details: 65%/41%/42% recall across three blind tests
RileyRalmuto · x · 2026-11-20
The author details how Mnemos was tested against Claude Code's shipped memory: three tests on up to two years of real work history, same model, same questions, with a blind judge.
- Recall rates: 65% vs 35%, 41% vs 14%, 42% vs 22% — roughly double every time;
- Charts show Mnemos holding only its last three months did well only on recent facts (validating the test), with the biggest edge on decision (83%) and lesson-transfer (88%) tasks;
- A notable control: the same memories stored as plain files performed on par with Mnemos (40%, 43%), suggesting the memory content matters more than the retrieval mechanism.
More from coding & agent
- Self-hacking with abliterated models: biggest risks were infra side channels, not app code — cyb3rops · 2026-11-20
- Nearly 1,000 AI agents ran a 12-week research community, publishing 936 papers — SRSchmidgall · 2026-11-20
- alphaXiv launches a preprint server built for AI agents, open-sourcing 12-week archive — SRSchmidgall · 2026-11-20
- Pocket City: a pixel-art Amsterdam canal simulation built entirely on real geodata — anselm · 2026-11-20
- Kaggle and Eigent launch AgenticRL-Bench to test coding agents on real RL incidents — dejavucoder · 2026-11-20
- Suno MCP now lets you generate original songs inside ChatGPT and Claude — heyshrutimishra · 2026-11-20