Letta launches FrontierMemory, a benchmark for agent learning-from-experience
dair_ai · x · 2026-10-22
Letta introduced FrontierMemory, a new benchmark that evaluates agent memory through downstream task performance. Unlike existing memory benchmarks focused on recall (already saturated via agentic search), FrontierMemory measures how agents learn from experience.
dair-ai's Omar Sanseviero highlighted it:
- Agent memory remains largely unsolved—for long-horizon work it's unclear which models, harnesses, memory systems, or memory types work best.
- Opus 5.5 topping the chart is no surprise, but open-weight GLM-5.3 performing so well on memory generation is exciting.
- His open question: how better memory and persistent agent capabilities translate into more performant, efficient RSI agents and harnesses.
The attached chart plots performance vs. average task cost (log scale) for GPT-6.1 Sol, Opus 5.5, Gemini 4 Argon, GLM-5.3, Kimi K3, Grok 4.7, and Muse Wonder.
Related event: Letta launches FrontierMemory benchmark for agent memory(3 posts)→
More from coding & agent
- Research Progress Graph: modeling authentic research trajectories, 35 agents found incomplete — ZhiyuChen4 · 2026-10-22
- Frontier models spark a comeback of quirky game-style personal homepages — HamelHusain · 2026-10-22
- Amp embeds interactive mini-apps in agent threads to replace typing input — HankYeomans · 2026-10-22
- Team-shared AI agent permissions: a Slack bot quietly hands everyone its database access — Careless_Love_3213 · 2026-10-22
- Qwen3.8 27B builds Mont-Saint-Michel voxel diorama in one hour from a single prompt — Teknium · 2026-10-22
- "My job is telling Claude to think bigger"—devs joke about AI ambition gap — paulnovosad · 2026-10-22