IBM’s Hierarchical BM25 serves 1B documents in 4.4GB and about 300ms per query
_reachsumit · x · 2026-08-04
IBM’s Hierarchical BM25 aims to make exact lexical retrieval practical at billion-document scale.
A flat BM25 index over one billion documents can take roughly 400 GB, with disk-backed serving stretching queries to 4–12 seconds. Hierarchical BM25 instead uses a resident coarse index to pick about 1,000 topical, size-balanced groups, then scores only the selected groups against the global statistics needed for exact BM25 scores. The approximation is limited to group selection, while returned scores remain identical to flat BM25. The system caps resident memory at about 4.4 GB, answers 16-term queries in around 300 ms, and sustains about 32 QPS with warm cache versus under 3 for flat indexing. The authors note that recall at billion scale and comparisons against BlockMax-WAND remain open.
More from Infra
- MiniMax H3 Text-to-Video Successfully Runs on a Single DGX Spark — Scobleizer · 2026-08-04
- Spectrum Acceleration for MiniMax H3 in ComfyUI: Up to 34% Lower Inference Time — marres · 2026-08-04
- Counterpoint Research: DRAM and NAND Memory Prices Expected to Peak in 2027 — SumitGup · 2026-08-04
- Atomic-scale logic circuits built from silicon dangling bonds showcased in new paper — teortaxesTex · 2026-08-04
- How a 5% Difference in Cache Hit Rate Spikes Token Cost 3-4x — Xianbao_QIAN · 2026-08-04
- Essential Features for Production LLM AI Gateways — Fun-Beginning5005 · 2026-08-04