Zero-Mem: Zero-Token Memory Operations Cut Agent Latency by 57.6%

rohanpaul_ai · x · 2026-08-14

The paper Zero-Mem introduces a novel long-term memory architecture for LLM Agents, featuring zero LLM token consumption for memory operations.

Zero-Cost Memory: Every memory retrieval and management operation outside the final QA requires no LLM calls, achieving zero input/output token usage.
Performance Boost: In controlled comparisons, this architecture reduced latency by 57.6% compared to the fastest baseline.
Technical Implementation: It preserves the original interaction history and builds two non-generative views over it: an entity-context graph for relationships and a temporal hierarchy for session and local context.
Deterministic Routing: At query time, deterministic routing decides which view to rely on, retrieves evidence, and calibrates the result before the final answer model generates the response.

This structured memory mechanism avoids turning memory management into a continuous generation problem, remaining faithful to raw interaction traces.

More from coding & agent