ANN search isn't nearly lossless either: MS MARCO-style benchmarks are small and too easy
srchvrs · x · 2026-10-26
In the same thread, srchvrs adds another critique of a common claim: people often say approximate nearest neighbor (ANN) search is nearly lossless versus exact k-NN, but benchmarks like MS MARCO are biased — small document counts and easily found documents containing direct answers mask the true cost of approximation.
This echoes the thread's broader point: small, stale benchmarks systematically understate degradation in frontier embedding and retrieval models.
Related event: Retrieval researcher challenges 'lossless' claims for MRL and ANN search(2 posts)→
More from Research
- Laying out DeepLIFT as a standalone computation graph: from sequence to attributions directly — jmschreiber91 · 2026-10-26
- Speculative Diffusion Paper Maps the Speed-Quality Pareto Frontier via Two Tradeoffs — UCLA · 2026-10-26
- Git Records Fall Short for Auditing AI's Role in Research Repositories, Paper Finds — Koutian Wu · 2026-10-26
- Fused DeepLIFT explained: why attribution hooks are cheap for Borzoi/AlphaGenome but hurt small models — jmschreiber91 · 2026-10-26
- New paper: AI agents negotiate badly under uncertainty and break deals on purpose — menhguin · 2026-10-26
- Cherimoya v0.3.1 Ships Fused DeepLIFT, ~7x Faster Attribution for Genomics Models — jmschreiber91 · 2026-10-26