Microsoft's xPRT: cross-passage pretraining boosts dense retrievers by 3.2%
_reachsumit · x · 2026-10-30
Microsoft researchers introduce xPRT (Cross-Passage Retrieval Pretraining), an unsupervised method that trains dense retrievers by predicting related passages a document's encoder never read, exploiting cross-passage structure.
- Evaluated on MTEB English v2 against five state-of-the-art objectives from a public unsupervised checkpoint, holding corpus and schedule fixed.
- xPRT outperforms the strongest baseline by 3.2% on average; many baselines regress in this continual-pretraining setting while xPRT yields the largest complementary gains.
- Offers a low-cost pretraining path for RAG and search systems where annotated query-document pairs are scarce.
More from Research
- UChicago seminar: hippocampal ripples, visual imagery BCIs, minimally invasive neural interfaces — plopesresearch · 2026-10-30
- Oopsie-data: Building the first open dataset of robot failure trajectories — chris_j_paxton · 2026-10-30
- Schmidhuber's 'Miraculous Year 1990-91': claiming roots of GANs, Transformers, pre-training — SchmidhuberAI · 2026-10-30
- NBER opens applications for Stanford Digital Economics and AI tutorial, due Nov 3 — avicgoldfarb · 2026-10-30
- FlexUMM shows unified multimodal models aren't so unified in embedding space — jindong_wang92 · 2026-10-30
- Data leakage hit 648 papers across 30 fields; autoarxiv now auto-reproduces arXiv papers — pandeyparul · 2026-10-30