Controlled test: multi-vector beats dense on 9 of 13 NanoBEIR benchmarks
tomaarsen · x · 2026-08-18
Does late interaction actually help? LightOn trained LateOn (multi-vector) and DenseOn (dense) on the same data with the same 149M ModernBERT backbone, differing only in pooling. Multi-vector wins 9 of 13 NanoBEIR datasets — 0.6868 vs 0.6764 mean nDCG@10 — with the same gap on full BEIR. In v6.0, every checkpoint format (PyLate, Stanford-NLP ColBERT, ColPali-style VLMs, or a bare backbone with a fresh projection) loads through one class, with prefixes, query expansion and the punctuation skiplist coming from the saved config.
More from coding & agent
- Agents turn conservative as context grows: edits shrink from 150 to 30 lines — dbreunig · 2026-10-16
- ai& becomes an official built-in Cline provider with 13 open-weight models — DavidBennett__ · 2026-10-16
- Open-source rizin-mcp lets LLMs run local binary and malware analysis via Rizin — Pure_Emergency1424 · 2026-10-16
- x402 as a free GTM channel: agents discover 100+ tools, no API keys needed — 0xJeff · 2026-10-16
- Sam Altman: stop getting better at prompts, build loops and graphs that prompt for you — goyalshaliniuk · 2026-10-16
- POV ski game on real Matterhorn elevation data, built with Claude Opus 5.5 — techartist_ · 2026-10-16