ColBERT deep dive: re-ranking can't recover the 18.6% of passages BM25 misses

antoine_chaffin · x · 2026-10-28

A new blog post compares ColBERT's two deployment modes: re-ranking BM25 results vs. end-to-end retrieval.

Key points:

2019 BERT rankers beat prior methods by 7 MRR@10 points on MS MARCO (8.8M passages), but score every candidate separately: 10,700ms per query on a V100 for 1,000 candidates.
ColBERT (Khattab & Zaharia, SIGIR 2020) encodes queries and documents separately with late interaction: MRR@10 of 34.9 vs. 34.7 for BERT-base, at 61ms per query — 170x faster at equal quality.
The blind spot: re-ranking can only reorder what BM25 hands it. BM25's top-1,000 contains the right passage only 81.4% of the time, so nearly 1 in 5 relevant passages is gone before ColBERT scores anything.
The fix is to stop relying on BM25 for candidates, leading into the end-to-end trade-offs the article explores.

More from coding & agent