Measuring a BM25 agent-memory index: 4.6 KB/entry RAM, linear, and where it breaks
Neither-Witness-6010 · reddit · 2026-10-27
The author measured their open-source in-process BM25 agent memory layer with tracemalloc (synthetic 30-word entries, Zipf vocabulary):
- 1,000 entries: 4.8 MB RAM / 1.7 MB disk / 7 ms search; 5,000: 23 MB / 34 ms; 10,000: 46 MB / 68 ms — 4.6 KB per entry in memory, 1.7 KB on disk, both linear.
Three non-obvious findings:
- Vocabulary is not the cost. It saturates near 3,000 terms; the cost is the per-entry TF-IDF dict held for every entry. A compact postings-list index would be far smaller.
- Search is a linear scan, not an inverted index — latency tracks N (170 ms at 25k). Fine for a few recalls per turn, wrong for query-heavy workloads.
- Persistence rewrites the whole JSON file per store, making insert cost quadratic — switch to append-only writes or a real database past a few thousand entries.
Extrapolation: 25k ≈ 115 MB, 100k ≈ 460 MB resident. Comfortable to 10k entries on a small box (the default cap, after which lowest-relevance entries are evicted); beyond that, embeddings or a proper index is the right answer. A 40-line benchmark script is shared; MIT licensed.
More from coding & agent
- Alchemy replaces long-lived API keys with OIDC for all AWS-GCP cross-cloud bindings — samgoodwin89 · 2026-10-27
- Testing Grok Bot: agent demands full-device GitHub PAT despite existing repo access — vincent_koc · 2026-10-27
- Pumpkin: open-source GUI manager gets llama.cpp running in 5 minutes, no terminal needed — gbolgurov · 2026-10-27
- OpenCode v2.2.0 adds desktop plugins and cache-friendly keyed session instructions — github-actions[bot] · 2026-10-27
- Fresh chat per task: 7 context engineering tips to avoid context rot — every · 2026-10-27
- Selenium vs Playwright vs AI testing: hybrid stacks are what survive real QA flows — kimjoshe8 · 2026-10-27