Training Recipe: Strongest Zero-Shot Model Beats General Retrievers
tomaarsen · x · 2026-08-26
The author shares the full recipe for training a strong zero-shot model: using a pre-supervised checkpoint, 1M domain pairs, in-batch negatives, full document length, and a higher-than-usual learning rate. The finetuned model achieves 84.9% rank-1 accuracy, significantly outperforming the zero-shot baseline. Multi-vector/late-interaction architectures are performing exceptionally well.
More from Apps
- Indie Game Prism Rules: Real-time Light Tracing with WebGL — measure_plan · 2026-08-27
- Grok Prompt: Generating Gigantic Forced-Perspective Ad Photography — aziz4ai · 2026-08-27
- Forensically: Free browser-based image forensics toolkit for detailed analysis — petewoodbridge · 2026-08-27
- Refine: AI verification tool trusted by world-class researchers — littmath · 2026-08-27
- Open Source AI Peer Review Tool 'coarse' Costs <$2, Outperforms Competitors — littmath · 2026-08-27
- .NET 11 Preview Released: C# 15 Union Types and MAUI CoreCLR — unixterminal · 2026-08-27