arXiv Study: Mixed Embedding Index Recovers Only 4.96 of 11.90 Points from Full Re-embedding
rohanpaul_ai · x · 2026-09-12
The arXiv paper 'Does Your Agent's Memory Survive a Model Upgrade?' (Ankit Goyal, Jaideep Ray) tests LC-RAW, RAG, compressed NOTES, and fixed-schema KG across 48 synthetic histories and two sub-10B open-weight models. KG-fixed barely shifts on a writer swap (+0.0004); NOTES swing +9.91/-13.28 points with 80% of deficit from information lost at construction time; a 50/50 mixed embedding index captures only 4.96 of an 11.90-point gain. Agents need memory compatibility tests, full re-embedding, and raw history retention.
Related event: Study Finds Agent Memory Doesn't Survive Model Upgrades Intact(2 posts)→
More from coding & agent
- Ollama and FriendliAI to Host Seoul Meetup on Building Production Agents with Open Models — ollama · 2026-10-28
- theo: If Your Product Requires a Dashboard, Claude Will Just Rebuild It — sull · 2026-10-28
- Magic Patterns' first engineer: Cursor in Slack, a Patterson review agent, repo clones over worktrees — Y Combinator Blog · 2026-10-28
- Dev builds AI job-search app for his wife, she lands a job, and he open-sources it — CurieuxExplorer · 2026-10-28
- Supercharging LLM classification with logprobs: soft classifiers you can threshold-tune — JnBrymn · 2026-10-28
- Codex daily build series day 23: skipping iCloud plugins for a reset — dotey · 2026-10-28