Skipping Retrieval and Cross-Encoding 10,000 Docs: BM25 Still Wins
CShorten30 · x · 2026-08-18
Weaviate's podcast "Drowning in Documents" shared a striking retrieval experiment: skip first-stage retrieval entirely and let a state-of-the-art cross-encoder score all 10,000 documents (gold plus random samples) itself — and old-school BM25 keyword search still came out ahead.
Mathew Jacob walks through the full-scoring setup and results, challenging the assumption that neural rerankers always beat classic sparse retrieval.
More from Research
- Ex-OpenAI researcher claims most neurons are legible now; critics call it hype — anshulkundaje · 2026-10-16
- Physicist pushes back on ML community's acceptance that models may be beyond understanding — tszzl · 2026-10-16
- Survey: AI agent evaluation is getting realistic but still fails to measure safety and reliability — burkov · 2026-10-16
- COVA-SFT dataset trains models to solve text problems with visual chain-of-thought — yoavartzi · 2026-10-16
- USTC paper uses speculative sampling to cut latency in interactive video generation — ustc · 2026-10-16
- Tsinghua's CoTranslator distills reverse CoT to boost 4B model math scores — Tsinghua · 2026-10-16