Practical guide to multi-vector embeddings: MaxSim retrieval and shrinking the index
tomaarsen · x · 2026-08-18
Tom Aarsen and two LightOn authors published a companion blog covering multi-vector (late-interaction) embedding models in practice:
- How it works: unlike dense models that compress text into one vector, multi-vector models keep a vector per token and score with MaxSim — alignment is non-lexical (live matches inhabit at 0.94).
- Gains vs costs: LightOn's LateOn (multi-vector) vs DenseOn on the same 149M ModernBERT backbone wins 9 of 13 NanoBEIR datasets (0.6868 vs 0.6764 mean nDCG@10), but the index is 16x larger (311.5 MB for 4,874 NQ passages).
- Shrinking: HierarchicalTokenPooling clusters token vectors with Ward linkage; poolfactor=2 halves the index at 100.6% of unpooled BEIR performance. Alternatives: a real late-interaction index or reranker-only use.
- Beyond text: ColQwen-Omni takes text, images, audio & video — retrieving recorded conversations zero-shot with no transcription; ColPali-style VLMs match text queries against page images, no OCR.
- Unified API: v6.0 loads PyLate, Stanford-NLP ColBERT and ColPali-style checkpoints through one class, with PyLate capabilities designed into ST alongside its authors.
More from coding & agent
- atuin v18.24 adds cross-device sync to resume AI threads across Claude and Codex — braelyn_ai · 2026-10-16
- Agents turn conservative as context grows: edits shrink from 150 to 30 lines — dbreunig · 2026-10-16
- ai& becomes an official built-in Cline provider with 13 open-weight models — DavidBennett__ · 2026-10-16
- Open-source rizin-mcp lets LLMs run local binary and malware analysis via Rizin — Pure_Emergency1424 · 2026-10-16
- x402 as a free GTM channel: agents discover 100+ tools, no API keys needed — 0xJeff · 2026-10-16
- Sam Altman: stop getting better at prompts, build loops and graphs that prompt for you — goyalshaliniuk · 2026-10-16