AML: A Unified Leaderboard for Evaluating Agent Memory Systems
AI寒武纪 · wechat · 2026-08-07
To address the lack of standardized evaluation for Agent long-term memory, over 20 academic institutions have jointly released the AgentMemoryLeaderboard (AML).
- Unified Evaluation: Integrates 10+ mainstream datasets covering text and code memory scenarios. It fixes the answering model and uses a multi-agent judging system to attribute score differences strictly to the memory system itself.
- Capability Profile: Breaks down scores into 7 dimensions (e.g., fact recall, multi-hop integration, temporal understanding, safety/privacy), providing an explainable diagnostic profile instead of a single mixed score.
- Low Barrier Entry: Participants only need to implement Add (write) and Search (retrieve) APIs. The platform handles answer generation, evaluation, and orchestration.
The first Agent Memory Challenge is currently underway, featuring separate tracks for open-source methods and commercial products, with results expected in mid-August.
Related event: 20+ Institutions Launch Agent Memory Leaderboard (AML)(5 posts)→
More from coding & agent
- Agent named 'Ilya' reportedly scores 1.1pt higher on benchmarks — daniel_mac8 · 2026-10-16
- Gemini CLI fix: substring matching inlined unrequested binary files into model context — priyanshu1976 · 2026-10-16
- Building AI agents on real healthcare claims data: the domain work that makes them work — alex_verem · 2026-10-16
- OpenAI stealth-testing mysterious internal model 'Aeon' in Codex desktop app and Dots — cedric_chee · 2026-10-16
- Vibe-coded iPhone Duo sniping bot built with Codex in 17 minutes, 41 tests passing — xiaohu · 2026-10-16
- Long-term multiplayer agent memory tool tops BEAM memory benchmark — CurieuxExplorer · 2026-10-16