AI Memory Retrieval Hit Only 22% With No Ranking — a Reranker Fixed It to 90%, No Model Calls
memokris · reddit · 2026-09-16
The author asked their internal AI memory system "what's my current job" and got an unrelated fact about a project named Henderson — the right answer was in the store but never retrieved.
Root cause: retrieval had no relevance ranking at all. It mapped the question to a topic and returned the eight most recent facts under it, whatever they said.
Measurement and fix: testing with 60 random facts, each fact's own text as the query:
- Before: 22% self-retrieval, 38% same-topic.
- After adding real ranking (candidate pool of 800, scoring by query coverage weighted by length): 90% and 97%.
- A closer-to-writing test using only the first 40% of a fact as the query: 15% → 88%.
The deceptive metric: the exact-match channel "hit" on 50 of 60 (83%), but only 13 of those 50 returned the target — "hit" and "returned the right thing" are different numbers, and the team had been watching the wrong one. Zero model calls, a few milliseconds of latency.
More from coding & agent
- Celesto 开源:给 AI 智能体一台安全持久的虚拟电脑 — aniketmaurya · 2026-09-16
- GPT-6 Astra turns a photo into a 3D game character via Hyper3D Rodin MCP — heyshrutimishra · 2026-09-16
- From Camera Roll to Boxing Game: GPT-6 Builds 3D Character via MCP — heyshrutimishra · 2026-09-16
- Gemini 3 dents Anthropic: OpenAI overtakes it in OpenCode usage and spending — justlikemedics · 2026-09-16
- NoSpoon agent cranks out hundreds of AI microdrama episodes daily before going internal — sachinmaya1980 · 2026-09-16
- Google open-sources ARTEMIS, a prompt-driven mobile automation tool with 99%+ success rate — alexcovo_eth · 2026-09-16