Scaling experiment: 10k pairs, 75 mins training, 1.2 points short of full run
tomaarsen · x · 2026-08-26
Experiments on an RTX 3090 show that training with only 100k pairs (10% of data) for 75 minutes yields NDCG@10 only 1.2 points lower than a full million-pair run, with most gains occurring in the first hour.
Comparisons reveal that Late Interaction models significantly outperform Dense models, beating the much larger Qwen3-Embedding-4B by 13 points. Notably, Qwen3's 8B version scores lower than its 4B counterpart.
Related event: Fine-tuning ColBERT on a single RTX 3090 beats general-purpose retrievers(14 posts)→
More from Research
- Cohere Labs world model course session 4: DreamerV1-V4 architectures in one go — Cohere_Labs · 2026-10-16
- Reasonable post-trains Nemotron 3.5 on 3,000 Verus proofs, lifting pass@1 ~30 points to 73.3% — fhuszar · 2026-10-16
- GPT-6.1 Sol audit finds 6% of arXiv papers had false main theorems, down to 1.5% recently — nitarshan · 2026-10-16
- One tiny word changes the conversation: how LLMs pick up on 'just' — StellaLisy · 2026-10-16
- SEAR: first large-scale controlled study of self-evolving robot agents, 7 LLMs, 330 tasks — xiye_nlp · 2026-10-16
- After Claude-assisted elliptic curve records fell, EC Rank challenge opens 156 record slots — Jsevillamol · 2026-10-16