Hybrid search isn't automatically better: dense embeddings beat BM25+RRF on all 3 held-out benchmarks
ashishllm · x · 2026-10-27
An AI engineer benchmarked BM25, dense retrieval (text-embedding-3-large), and Qdrant RRF hybrid search on ArguAna, SciFact, and FinRank:
- Dense won held-out NDCG@10 on all 3 datasets
- Tuned RRF beat dense on 2/3 tuning sets but lost on all 3 held-out sets
- Hybrid can add coverage yet hurt rank-1 quality
The takeaway for AI engineers: hybrid search isn't automatically better — run your own experiments with held-out evaluation instead of following industry defaults.
More from Research
- Eric Topol highlights immunome paper combining multi-assay data with AI to model immune aging — EricTopol · 2026-10-28
- Google Research's 'sleep framework' lets LLMs keep learning, retaining up to 10M tokens — behrouz_ali · 2026-10-28
- CATCH: ~200 examples of adversarial training make LLMs robustly honest across benchmarks — irinarish · 2026-10-28
- Andrew Critch's dubious credences: P≠NP at 80%, Riemann hypothesis at just 60% — AndrewCritchPhD · 2026-10-28
- AI-driven Indian biotech PopVax, backed by $20M from Vitalik's Balvi fund, enters human trials with fridge-stable broad-spectrum Covid vaccine — chaitjo · 2026-10-28
- Cognition launches FrontierCode 2.0, a coding benchmark testing judgment beyond the spec — DevinAI · 2026-10-28