Agent memory benchmarks measure recall, but provenance is what bites in production

Mr_Daud_9899 · reddit · 2026-11-18

A developer running a continuously-operating agent with cross-session memory shares a postmortem: memory benchmarks almost all measure recall, but none measure provenance — the part that fails in production.

The failure mode: the agent stored an assumption early on, acted on it weeks later when the original context was gone. Retrieval was fine — the store returned exactly what was written; nothing recorded when it was learned, from what source, or whether it had been contradicted since.

Three things to build in from day one:

He also asks the community whether provenance actually survives into context assembly at inference time.

Original post →

More from coding & agent

coding & agent channel →