Fine-tuned multi-vector retrieval beats dense and sparse models, benchmarks show
tomaarsen · x · 2026-08-26
tomaarsen summarizes his blogpost: fine-tuning models for your own task pays off, and multi-vector (MVE) models are very strong, beating every dense or sparse retrieval model he could find. Two off-the-shelf models scored 67.04 and 61.42, both underperforming. He also notes BM25 looks unusually strong on MIRIAD because its queries were generated from the documents, giving lexical matching a huge advantage. Takeaway: for solid search, fine-tune an MVE model.
Related event: BM25 Beats Some LLM Retrievers on MIRIAD, Sparking Debate(3 posts)→
More from Research
- Pathway CSO pitches BDH as post-Transformer architecture uniting memory, reasoning, adaptation — schwentker · 2026-10-15
- EmbodiedEvo tops RoboDojo at 38.95% success by letting robots inherit failed practice — thetripathi58 · 2026-10-15
- 797GB of n-gram counts in CPU RAM let 1B models match 57% more compute — PangWeiKoh · 2026-10-15
- Google's HARNESS² evolves agent harnesses without verifiers, up to +11.5 points on WorkBuddy-Bench — dair_ai · 2026-10-15
- Subbarao Kambhampati to dissect the 'mythos of LLM thinking tokens' at UMN — rao2z · 2026-10-15
- How 3D Gaussian Splatting gives robots spatial memory of what's out of view — TheTuringPost · 2026-10-15