Vector DBs for Retrieval, Not Facts

recro69 · reddit · 2026-07-11

The author argues that vector databases are great as a retrieval layer, but not suited to be the "source of truth" for business systems.

They point out that many business AI agents store long-term memory in vector DBs, which causes issues if the information changes:

When querying things like "Is the order complete?", "Has the invoice been paid?", "Is the approval passed?", or "Is inventory available?", the system needs the latest status, not semantically similar old information.
Vector retrieval excels at finding "relevant content" but cannot guarantee "current correctness".
Embeddings aren't necessarily wrong; what's wrong is treating them as the source of business truth.

The author suggests a tiered architecture:

Structured databases: Store factual states like orders, permissions, balances, bookings, and approval flows
Vector databases: Store semantic context like documents, policies, customer service records, meeting minutes, product manuals, and knowledge bases
LLMs: Synthesize both to perform reasoning

They conclude with a counterfactual question: If your vector database disappeared tomorrow, could your business AI agent still correctly answer business questions? If not, it has been treated as the "source of truth".

Original post →

Vector DBs for Retrieval, Not Facts

More from coding & agent