Measured: in-process BM25 memory costs ~4.6KB RAM per entry, linear growth
Neither-Witness-6010 · reddit · 2026-11-06
The author benchmarked a local agent memory layer using in-process BM25 + SQLite (no vector DB): 1k entries cost 4.8MB resident RAM with 7ms search; 5k → 23MB/34ms; 10k → 46MB/68ms—linear at 4.6KB RAM and 1.7KB disk per entry.
Key findings:
- Vocabulary isn't the cost: it saturates around 3,000 terms; the real cost is the per-entry term→float dict holding TF-IDF vectors—compact postings lists would be far smaller.
- Search is a linear scan, not an inverted index: latency tracks N (170ms at 25k), fine for a few recalls per turn, wrong for query-heavy workloads.
- Persistence rewrites the whole JSON file per store, making insert cost quadratic—first thing to fix past a few thousand entries.
Extrapolation: 25k ≈ 115MB, 100k ≈ 460MB RAM; the backend caps at 10k entries and evicts lowest-relevance items beyond that. Verdict: comfortable for a personal agent's history up to 10k entries; beyond that, embeddings or a real index. Author open-sourced CogniCore (MIT) with a 40-line reproducible benchmark script and asks for equivalent numbers from embedding-backed stores.
More from coding & agent
- Exo: an agent harness that improves itself at runtime, slashing cost and time to SOTA solutions — berkeley_ai · 2026-11-06
- Conductor founder joins Meta, 134 AI coding-agent alternatives flood X to fight for its users — jasonkneen · 2026-11-06
- Anaconda demos CLI + MCP with Kilo Code: managing conda environments via natural language — anacondainc · 2026-11-06
- Search fails new hires: project homes let teams find things they can't yet search for — dSebastien · 2026-11-06
- Running a full experiment suite from a phone via Cursor cloud agents — malliktwts · 2026-11-06
- 10 days with Bend2: migrating core TypeScript logic to functional style with laws and proofs — mattpocockuk · 2026-11-06