Sentence Transformers v6 adds multi-vector late-interaction retrieval with ColBERT support

tomaarsen · x · 2026-08-19

Hugging Face published a blog introducing the fourth model type in Sentence Transformers v6.0: MultiVectorEncoder, supporting ColBERT-style late interaction retrieval.

Any PyLate checkpoint and Stanford-NLP ColBERT checkpoint loads directly, and colpali-engine models for visual document retrieval work through the same familiar API as dense, sparse, and reranker models.
Unlike a regular embedding model that compresses text into one vector, a multi-vector model keeps one vector per token and scores query–document pairs with the MaxSim operator, preserving token-level matching information — usually stronger retrieval at the cost of a bigger index.
It is also state of the art for visual document retrieval, matching text queries directly against page images with no OCR step.

The tutorial covers loading various checkpoint formats, encoding and scoring, plugging into a search stack, running on page images, and keeping the index affordable — all via pip install -U sentence-transformers.

Original post →

Sentence Transformers v6 adds multi-vector late-interaction retrieval with ColBERT support

More from coding & agent