NLI alone rejects 5/6 new agent memories; a 0.45 similarity gate fixes it all

Alternative_Pin9598 · reddit · 2026-10-26

A developer's post-mortem on validating agent memory writes when multiple writers (chat, CRM sync, scraper, other agents) share one store.

The failure

Checking each incoming fact against all memories with a local DeBERTa-v3-small NLI model rejected 5 of 6 facts — including two genuinely new ones. NLI is a pairwise model and confidently flags unrelated sentences as contradictions ("learning cello" vs "job in logistics" scored 1.000).

The fix: gate NLI by similarity

Real conflicts sat at 0.50–0.74 cosine similarity; false ones at ≤0.41. Running NLI only on pairs above 0.45 sorted all six facts correctly (3 CONFLICT, 1 ALREADYKNOWN, 2 NEW). A full SQL query implementation is provided.

Caveats

Narrow margin (0.41 vs 0.50): tune the threshold on your own memories.
Conflict ≠ error: route by writer trust — user-first statements supersede, low-trust writers get rejected, equal-trust conflicts go to a human.
The gate enforces consistency, not truth; a wrong stored memory will flag the correct new fact.
Keep memories short.

Original post →

NLI alone rejects 5/6 new agent memories; a 0.45 similarity gate fixes it all

More from coding & agent