Study Finds Embedding Models Often Ignore Retrieval Instructions; Distractor Fine-tuning Helps

_reachsumit · x · 2026-10-08

Researchers from the University of Turku released an arXiv paper, Your Prompt Should Do More, examining how prompted embedding models process retrieval instructions in asymmetric retrieval.

Key finding: when the candidate pool includes distractor texts similar to the query, embedding models often fail to follow even simple task instructions — the detailed retrieval prompt is effectively ignored.
Mechanism: the authors attribute this to current training setups and evaluation protocols, which lack query-side distractors and thus never force models to actually use the instructions.
Fix: fine-tuning with added query-side distractors yields substantial improvements while minimally affecting other tasks.

Authors: Amanda Myntti, Jenna Kanerva, Veronika Laippala, Filip Ginter.

Original post →

Study Finds Embedding Models Often Ignore Retrieval Instructions; Distractor Fine-tuning Helps

More from Models