Weaviate Podcast with Mathew Jacob: More Retrieval plus Reranking Isn't Always Better

Weaviate Podcast #141 features Mathew Jacob, author of the Databricks paper "Drowning in Documents," which challenges the popular RAG intuition of retrieving more documents and then reranking. The core finding: simply increasing retrieval depth often does not help; reranking is the key lever, yet adding reranking does not always help and can even hurt performance.

Confirmed

• Jacob completed the paper at Databricks and is now a PhD student in ML systems at the University of Washington.
• Core finding: more retrieval alone rarely improves RAG quality; reranking is critical, but "retrieve more + rerank" does not always yield better results and can degrade performance.
• The discussion covered how rerankers such as cross-encoders work and when they apply.
• A key counterintuitive result: Recall@10 declines when more than roughly 100 documents are pushed to a cross-encoder reranker.
• A striking experiment: skipping first-stage retrieval and letting a state-of-the-art cross-encoder score all 10,000 documents (including the gold-standard ones), the classic keyword retriever BM25 still came out ahead.

Why it matters

• "Retrieve more, then rerank" is one of the most common RAG design patterns; if its gains are capped or can backfire, this directly affects production cost and latency trade-offs.
• Results like the Recall@10 drop beyond 100 documents and cross-encoder full reranking losing to BM25 caution against blindly trusting rerankers—developers should benchmark rather than follow intuition.
• The episode offers experiment-backed guidance from frontline research for deciding retrieval depth and whether to add a reranker.

2026-08-17 ~ 2026-08-18 · 9 related posts

• [source] Weaviate Podcast: Why "Retrieve More, Rerank" might be wrong — CShorten30 · 2026-08-17
• Weaviate Podcast #141: Drowning in Documents author on why retrieving more and reranking isn't always better — CShorten30 · 2026-08-17
• [source] Weaviate Podcast: Misconceptions in retrieval and reranking with Databricks author — CShorten30 · 2026-08-17
• Why "Retrieve More, Rerank" is Wrong? Deep Dive on Drowning in Documents Paper — CShorten30 · 2026-08-17
• Reranking Paradox: Performance Drops as Document Count Increases — CShorten30 · 2026-08-18
• Weaviate Podcast: Why retrieving more and reranking isn't always better — CShorten30 · 2026-08-18
• [source] Skipping Retrieval and Cross-Encoding 10,000 Docs: BM25 Still Wins — CShorten30 · 2026-08-18
• Reranking Deconstructed: Weaviate Podcast on 'Drowning in Documents' Insights — CShorten30 · 2026-08-18

1 near-duplicate retellings: CShorten30