Controlled test: multi-vector beats dense on 9 of 13 NanoBEIR benchmarks
tomaarsen · x · 2026-08-18
Does late interaction actually help? LightOn trained LateOn (multi-vector) and DenseOn (dense) on the same data with the same 149M ModernBERT backbone, differing only in pooling. Multi-vector wins 9 of 13 NanoBEIR datasets — 0.6868 vs 0.6764 mean nDCG@10 — with the same gap on full BEIR. In v6.0, every checkpoint format (PyLate, Stanford-NLP ColBERT, ColPali-style VLMs, or a bare backbone with a fresh projection) loads through one class, with prefixes, query expansion and the punctuation skiplist coming from the saved config.
Related event: Sentence Transformers v6.0 ships with first-class late interaction models(33 posts)→
More from coding & agent
- Using AI Agents to draft release reports from evidence collections — CodeByPoonam · 2026-08-19
- Netlify integrates OpenRouter to enable model swapping without code changes — thisiskp_ · 2026-08-19
- Dev runs three Codex accounts plus Claude to parallelize coding agents — ChanceKelch · 2026-08-19
- DeepSeek open sources 'deepseek-harness' agent framework with 130k+ stars — alex_verem · 2026-08-19
- Cloud agents save hours vs local setups: a dev's parallel-workflow experience — brandon_galang · 2026-08-19
- IBM Bob resolves 60.6% of SWE-bench Verified at a median $1.05 per task — TejasKumar_ · 2026-08-19