Graph memory underperforms flat retrieval on LongMemEval benchmarks
omarsar0 · x · 2026-09-02
A study evaluates whether graph memory outperforms flat retrieval for long-term agents. On LongMemEval, the graph approach achieved a Token F1 of 0.42, lagging behind the flat vector baseline's 0.47. Paired bootstrap testing confirmed a significant gap (95% CI -0.085 to -0.016). The performance hit is concentrated on questions requiring specific assistant turns, where accuracy dropped from 0.911 to 0.607 because splitting turns into entities discards necessary surface forms. However, the forgetting module performed well, successfully pruning 9.8% of nodes based on recency and access frequency.
More from coding & agent
- How to Prevent Skill Drift When Iterating on AI Prompts? — petergyang · 2026-09-02
- Not Diamond releases model routing method, cuts costs 20-80% — rohanpaul_ai · 2026-09-02
- AgentFold: Closed-Loop Agentic Search for Protein Folding Model Design — rohanpaul_ai · 2026-09-02
- Claude Fable 5.1 crushes hard coding benchmarks, outpaces Chinese models — minchoi · 2026-09-02
- Claude Fable 5.1 builds subway FPS game with Ultracode — minchoi · 2026-09-02
- Claude Fable 5.1 one-shots a Mario Kart-style game — minchoi · 2026-09-02