Reranking more docs backfires: Recall@10 peaks near 100, then declines, study finds
CShorten30 · x · 2026-09-15
Weaviate Podcast #141 features Mathew Jacob's paper "Drowning in Documents": as a Databricks intern he found that scaling cross-encoder reranking depth lifts Recall@10 only up to 100 documents, after which it declines — a real, counterintuitive curve he initially mistook for a bug. The work proposes sliding-window listwise reranking (robust at 1,000 docs) and ranking cascades, and the episode also covers TraceLab, a dataset of 40,000 real coding-agent traces.
More from coding & agent
- User has Grok bot schedule all Pinterest posts with auto-generated titles — prasenx · 2026-09-15
- Work Louder brings all Codex Micro keyboard magic to Creator Micro 2 — LukeW · 2026-09-15
- tldraw's AI design sprint: Codex prototypes every discussed interaction overnight, artifacts by day 2 — max__drake · 2026-09-15
- Research agents log what they cite — should they also defend what they reject? — Entire_Mark8010 · 2026-09-15
- Grok Bot Auto-Schedules Pinterest Posts, Writes Titles From Images—Where Claude Failed — prasenx · 2026-09-15
- Amplitude tripled PR volume in six months by fixing CI, not agents — mobileraj · 2026-09-15