Agent memory benchmarks measure recall, but provenance is what bites in production
Mr_Daud_9899 · reddit · 2026-11-18
A developer running a continuously-operating agent with cross-session memory shares a postmortem: memory benchmarks almost all measure recall, but none measure provenance — the part that fails in production.
The failure mode: the agent stored an assumption early on, acted on it weeks later when the original context was gone. Retrieval was fine — the store returned exactly what was written; nothing recorded when it was learned, from what source, or whether it had been contradicted since.
Three things to build in from day one:
- Every memory write carries a source and timestamp, append-only; corrections are new rows, not overwrites
- Retrieval returns provenance alongside the fact, so the agent can say "I believe X, recorded on date Y from source Z"
- A way to diff memory between two points in time, not just see current state
He also asks the community whether provenance actually survives into context assembly at inference time.
More from coding & agent
- Replit launches Missions, an agent-swarm harness that hits your goals within a set budget — amasad · 2026-11-18
- YC Paper Club: why the harness matters more than the model — eherrerosj · 2026-11-18
- AI-built Paris replica streams city blocks in real time with historically accurate traffic — Dimillian · 2026-11-18
- Reflection launches Trusted Access Program, starting with autonomous pentest tool Laser — brandondamos · 2026-11-18
- Tiny 2.6B local model runs an AI town, with every decision captured via Otel in Agent Lens — coreweavedevs · 2026-11-18
- Vercel CEO: Define spec, exit criteria and outcomes before you generate code — evilrabbit_ · 2026-11-18