A vector DB is not LLM memory: long thread maps write/read paths, failure modes and evals

techNmak · x · 2026-10-16

techNmak published a detailed thread debunking the idea that a vector database equals an LLM memory system — it's just one possible component. Persistent memory requires storage, retrieval, state management, context selection, and evaluation.

Key structure:

Write path: Extract → Validate → Reconcile → Persist. Records need identity, scope, provenance, timestamps, and versioning; updates must preserve relevant history without keeping conflicting facts as equally current.
Read path: Authorize → Lookup/Search → Select/Rank → Assemble Context → Generate. Retrieval can use SQL, keyword, vector similarity, hybrid, or graph traversal; similarity scores alone don't establish correctness, freshness, or authorization.

The thread distinguishes mechanisms often conflated as "LLM memory": model weights, active context, application-managed persistent memory, and KV cache (which reduces computation, not storage needs). RAG and fine-tuning don't automatically handle memory updates, conflict resolution, or retention policies.

The accompanying infographic covers memory lifecycles, semantic/episodic/procedural memory roles, context budgeting, stage-by-stage failure tracing (never saved, saved but not found, found but excluded, misused), and evaluation beyond retrieval accuracy — testing recall, updates, temporal ordering, cross-session consistency, access isolation, and abstention.

Original post →

A vector DB is not LLM memory: long thread maps write/read paths, failure modes and evals

More from coding & agent