Pooling-aware training keeps 97.8% retrieval quality with 4x fewer vectors
tomaarsen · x · 2026-11-11
tomaarsen, author of Sentence Transformers, trained two ModernBERT-base models on the same 120k MS MARCO triplets, changing only the loss.
- Multi-vector models store a vector per token (great for retrieval, costly to store); token pooling merges similar document vectors into cluster means
- With pooling-aware training, 49 vectors per document hits NanoBEIR@13 mean nDCG@10 of 51.23 vs 49.03 for standard training — 97.8% of its own unpooled score, vs 94.6%
- The chart shows pooling-aware training consistently beats standard training at every pooling level (32/48/64/96 vectors)
More from Research
- ArcGrav: a cross-model Agent Control Language to cut token overhead — ArchitecturalGravity · 2026-11-12
- Open-source dual-arm robot solves tangram puzzles with 90 demos, ~60% success — DanGrover · 2026-11-12
- FutureHouse's deslopper trains Kosmos to write indistinguishably from human experts — proteinrosh · 2026-11-12
- Fleuret amplifies debate: AI can't slash wet-lab experiments in unmapped biology — francoisfleuret · 2026-11-12
- Recursive's Eureka Machine ships Flint: a sparse 40B model with 2.5B active for local agentic coding — agihouse_org · 2026-11-12
- Recursive bets on open-endedness: agents running massive parallel experiments like scientist communities — agihouse_org · 2026-11-12