14.5 hours on one RTX 3090: finetuned multi-vector retriever beats all general-purpose models
tomaarsen · x · 2026-08-26
Tom Aarsen, author of sentence-transformers, published an in-depth tutorial: v6.0 introduces a fourth model type, MultiVectorEncoder, for ColBERT-style late interaction retrieval with a complete training pipeline. Everything runs via pip install -U "sentence-transformers[train]".
The post walks through each component of finetuning multi-vector models — model, datasets, losses, training arguments, evaluators, and the trainer — with practical examples.
For evaluation, his multi-vector-encoder/mLateOn-medical, trained in 14.5 hours on a single RTX 3090, easily outperforms every general-purpose retrieval model he could find on his medical retrieval benchmark: dense, sparse, lexical, and multi-vector alike. The method also trains strong new multi-vector models from scratch.
More from coding & agent
- PrimeIntellect verifiers v0.3.1: Model Interception and Persistent ACP Sessions — xeophon · 2026-08-27
- RAG Isn't Dead: Navigating Retrieval vs. Agentic Search — hugobowne · 2026-08-27
- Devin rebuilt its renderer for massive sessions — premqnair · 2026-08-27
- Karpathy's 1-Hour Stanford Lecture: From LLM to Prompt to Agent to Graph — AlishaOutridge · 2026-08-27
- SpaceXAI engineer shares guide on building a 24/7 Agent team — soleio · 2026-08-27
- LiveKit builds patient intake agent end-to-end on Grok voice models with ZDR — SpaceXAI · 2026-08-27