MemCredit: Semantic Relevance Fails at Agent Memory Selection, Paper Finds
omarsar0 · x · 2026-11-14
Researchers from UESTC and NTU propose MemCredit, shifting agent memory management from semantic relevance to decision utility. Key finding: most memories retrieved by semantic similarity have zero effect on the agent's next action, and relevance correlates poorly with usefulness.
- Method: Controlled counterfactual interventions measure each memory's marginal contribution (does the agent's action change with/without it?), then distill these credits into a lightweight utility estimator. Relevance still recalls candidates; the estimator decides what fits the token budget—no agent policy changes or extra environment interactions needed.
- Results: On ALFWorld with a 500-token memory budget, MemCredit reaches 86.57% success vs 65.67% for BM25 retrieval, using 152.1 memory tokens per decision (74.5% fewer than BM25).
- Counterfactual analysis shows memory utility is sparse, strongly state-dependent, and poorly captured by semantic relevance.
Takeaway: decision utility is a more effective principle than relevance alone for long-horizon agent memory.
More from coding & agent
- Letting LLMs write a tiny DSL instead of JSON tool calls: is it worth it? — bye_ella · 2026-11-14
- Train system design requirement-discovery with AI using vague-problem drills — anirudhology · 2026-11-14
- Dev with zero KiCad experience gets Codex to do PCB design in half a day — mckbrando · 2026-11-14
- One prompt, one 97KB HTML file: a procedural swimmable underwater world built by Claude — Any-Reputation8118 · 2026-11-14
- AG-UI, the agent-user interaction protocol for frontend apps, hits 17k GitHub stars — ag-ui-protocol · 2026-11-14
- Tencent open-sources LoopForge to self-host coding-agent workflows with approval gates — Tencent · 2026-11-14