SYNTRA self-improving data synthesis scales embedding models across 15 domains
_reachsumit · x · 2026-10-14
SYNTRA is a new retrieval data synthesis method that adapts embedding models to new tasks using only an unlabelled corpus and a few query-document pairs.
Mechanism:
- A self-improvement loop iteratively refines reusable synthesis instructions based on feedback on trial outputs
- Eliminates costly manual data design
Results:
- Validated on 24 retrieval datasets across 15 domains
- Improves widely used strong embedding models both with and without existing labelled training sets
- Self-improvement boosts retrieval on all six evaluated tasks, beating human calibration on four
- Performance on MS MARCO keeps improving as training data scales to 32x initial size
Code and data are open-sourced.
Related event: SYNTRA Enables Label-Free Cross-Domain Expansion of Embedding Models(2 posts)→
More from Research
- Fingerprinting Leaderboard benchmarks model provenance as copying takes minutes vs millions to train — omarsar0 · 2026-10-14
- Is today's agent memory stack just a workaround for model architecture? — Sun-Tzu-Maybe · 2026-10-14
- Persona experiments on Talkie, a 13B LLM trained only on pre-1931 text — dhadfieldmenell · 2026-10-14
- HKU and Tencent open-source Multics, a multi-player world model that trains on 4 players but scales to 10 — Scobleizer · 2026-10-14
- SAP paper teaches LLM agents to predict their own failure (AUROC 0.85) — SAP · 2026-10-14
- CUHK's Orchestra robot framework completes 7.35-step subtask sequences — CUOHK · 2026-10-14