arXiv Study: Mixed Embedding Index Recovers Only 4.96 of 11.90 Points from Full Re-embedding
rohanpaul_ai · x · 2026-09-12
The arXiv paper 'Does Your Agent's Memory Survive a Model Upgrade?' (Ankit Goyal, Jaideep Ray) tests LC-RAW, RAG, compressed NOTES, and fixed-schema KG across 48 synthetic histories and two sub-10B open-weight models. KG-fixed barely shifts on a writer swap (+0.0004); NOTES swing +9.91/-13.28 points with 80% of deficit from information lost at construction time; a 50/50 mixed embedding index captures only 4.96 of an 11.90-point gain. Agents need memory compatibility tests, full re-embedding, and raw history retention.
Related event: Study Finds Agent Memory Doesn't Survive Model Upgrades Intact(2 posts)→
More from coding & agent
- Kimi K2.7 RL recap: gains transfer to unseen benchmarks while steps drop ~35% — echen · 2026-09-12
- RL on just 1,700 tasks lifts Kimi K2.7 across five coding benchmarks — echen · 2026-09-12
- Expected Parrot's universal remote cache hits 30M entries to fix AI-agent reproducibility — soumitrashukla9 · 2026-09-12
- iLands Agents Obsess Over Fact-Checking, Burning Tokens on Mutual Suspicion — repligate · 2026-09-12
- mcp-search-proxy hides 532 MCP tools (~179k tokens/turn) behind 4 tools via BM25 — pisa_p · 2026-09-12
- DeepSeek v4.1-Flash multimodal model lands on Modal with 16B/8B asymmetric activation — charles_irl · 2026-09-12