Reranking Paradox: Performance Drops as Document Count Increases
CShorten30 · x · 2026-08-18
This post highlights a counterintuitive technical phenomenon in Retrieval-Augmented Generation (RAG): while the common workflow is "retrieve more documents -> rerank -> get better results," experiments show that Recall@10 actually plummets when pushing more than 100 documents to a Cross-Encoder reranker.
The reason is that rerankers act less like stronger retrievers and more like "boosting," amplifying the errors of the first-stage retriever. When input noise (irrelevant documents) becomes too high, the reranker is prone to misjudgment.
More from Infra
- TRL reproduces Modal's mixed-precision async RL: plain GRPO collapses, SC+MIS holds 0.72 — QGallouedec · 2026-10-16
- Hugging Face scales Indic pretraining with 2.8T tokens and an open data pipeline — Hugging Face Blog · 2026-10-16
- NVIDIA Vera Rubin NVL72 hits up to 5.7x GB300 throughput per MW on agent benchmark — ArtificialAnlys · 2026-10-16
- 14 open-source LLM observability tools to know in 2026, from Langfuse to Laminar — TheTuringPost · 2026-10-16
- Local AI is a routing problem, not a privacy feature: where each task should run — bigdata · 2026-10-16
- Moving agent work from GPUs to CPUs for durable, predictable actions — cramforce · 2026-10-16