Mnemos memory benchmark hits 41% on 2-year corpus, beating Claude's 21% native recall
RileyRalmuto · x · 2026-10-12
Developer RileyRalmuto shared the final pre-release benchmark for Mnemos, his memory project, tested on a 2-year corpus of 2,000 memories with search tools disabled for fairness:
- Mnemos: 41%, 57s runtime
- Plain text files: 40%, 90s, 5.3x usage
- claude.ai native memory: 21%
- Claude Code native memory: 14%
- No-memory baseline: 7%
Mnemos matched plain files on accuracy while costing 5x less and running 1.6x faster (both figures slightly inflated without search). When Claude systems failed, the answer was almost always in files they'd already read—they just missed it.
Acknowledged gaps: recent memories get buried in large stores, and most of Mnemos's 8 misses were outdated facts. A final test with full search tools on real-world conditions (1-month corpus) is underway.
More from coding & agent
- What should personal agents do first? Boring renewal and expiry reminders — sujingshen · 2026-10-12
- Rive renders entire 3D city with vector graphics, ships CLI for Codex and Claude Code agents — soleio · 2026-10-12
- TechnoLama shows the AI-recreated PowerPoint lookalike from the ArtCraft suite — technollama · 2026-10-12
- Unpacking agent product forms: capability, collaboration and accountability are separate axes — sujingshen · 2026-10-12
- Dev vibe-codes a minimal Mac app that floats YouTube over everything — blakesamic · 2026-10-12
- Epic Ships Official Unreal Engine MCP, Letting AI Agents Drive UE Projects — Kyrannio · 2026-10-12