Why tiny encoders hold up: multi-vector retrieval trades storage for quality
tomaarsen · x · 2026-10-26
tomaarsen explains why MoEdernColBERT's tiny encoders still score well: multi-vector retrieval keeps a 128-d vector per token, and MaxSim lets query tokens match different document tokens. That means more vectors to store and score than one dense vector per text — the quality comes with real storage and scoring overhead.
Related event: VAGOsolutions releases tiny 1.5M-115M MoE ColBERT retrieval models(10 posts)→
More from Models
- China Telecom's XingChen lab open-sources Xing4.0-29B-Intent, a 29B MoE with 4B activated params — FellMentKE · 2026-10-26
- Deleted vLLM PR Hints Qwen4-27B Uses 4-Way Parallel Residual Streams (Unconfirmed) — Ok-Shower7286 · 2026-10-26
- a16z: OpenRouter token demand up ~27x in a year, open-weight models now serve 72% — TheZachMueller · 2026-10-26
- Running a 125B model on 8GB VRAM with Strata: 900 t/s prompt processing, 30 t/s generation — BleynSpecnaz · 2026-10-26
- More OG Hugging Face profiles to follow: local AI, diffusion, kernels and transformer.js — ariG23498 · 2026-10-26
- 9 Hugging Face profiles worth following, from TRL agents to vLLM kernels — ariG23498 · 2026-10-26