A plain SQLite FTS5 baseline beat the median on the Agent Memory Leaderboard
No_Issue_8224 · reddit · 2026-10-21
On the Agent Memory Leaderboard's August public text board, SQLite-FTS-Baseline scored 41.79, ranking 22 of 50 — above the 40.78 median. It's a minimal deployable baseline: SQLite stores raw messages, FTS5 handles full-text retrieval, BM25 ranks. On the same board, just-a-BM25 scored 38.47 (rank 35), running BM25 within exact userid on verbatim messages with no LLM, embedding model or reranker.
The author's takeaway: a baseline is something you actually run, not a number you assume. Before adding retrieval machinery to a local assistant — whose history mixes exact names, commands, paths and old requests — measure how often plain search finds the intended message and whether anything extra improves the same queries. Otherwise you're mostly collecting pleasant demos.
More from Research
- EMNLP paper: diffusion LMs internally encode latent denoising timestep representations — gsarti_ · 2026-10-23
- KU Leuven and VIB open two Group Leader professorships in AI for Biology, start 2027 — jpillowtime · 2026-10-23
- Subspace Tuning curbs LLM catastrophic forgetting by freezing principal singular directions — scottniekum · 2026-10-23
- Perceptic launches PercepticOS, an AI workspace for 'scientific superintelligence' — nathanbenaich · 2026-10-23
- EPFL's Bosselut to talk long context as test-time learning at ELLIS seminar — andre_t_martins · 2026-10-23
- DRATS: Adaptive Task Sampling Boosts Worst-Task Performance in Multi-Task RL — Cohere · 2026-10-23