From ModernBERT-base to 0.48 NanoBEIR nDCG@10 in 25 Minutes on One RTX 3090
tomaarsen · x · 2026-08-18
Training late-interaction models in v6.0 works like other model types: 4 new losses (including cached & distillation variants), 5 new evaluators, and a Trainer taking the same arguments you already know. From a bare ModernBERT-base, NanoBEIR mean nDCG@10 improves from 0.1338 to 0.4831 in 25 minutes on a single RTX 3090. Sentence Transformers also doesn't need its own late-interaction index: these indexes store whatever encodedocument returned — Qdrant, Weaviate, Vespa, LanceDB, VectorChord & Milvus index multi-vectors natively, and LightOn's fast-plaid is a pip install away.
More from coding & agent
- Agents turn conservative as context grows: edits shrink from 150 to 30 lines — dbreunig · 2026-10-16
- ai& becomes an official built-in Cline provider with 13 open-weight models — DavidBennett__ · 2026-10-16
- x402 as a free GTM channel: agents discover 100+ tools, no API keys needed — 0xJeff · 2026-10-16
- Sam Altman: stop getting better at prompts, build loops and graphs that prompt for you — goyalshaliniuk · 2026-10-16
- POV ski game on real Matterhorn elevation data, built with Claude Opus 5.5 — techartist_ · 2026-10-16
- Poisoned SERPs broke rank tracking — a Claude MCP + Notion workaround — VeryWellVersed · 2026-10-16