Dense Retrievers Fail When Instructions Just Reverse a Relation's Direction
_reachsumit · x · 2026-11-13
A new paper, When Direction Matters, exposes a specific weakness in dense retrievers' instruction following: they perform well when lexical or attribute cues distinguish the target, but become unreliable when relevance hinges on a directional relation over the same entity pair.
The authors introduce SwitchIR, contrasting Constraint Shift and Explicit Filtering with Relational Flip, where query and documents stay fixed while the requested relation direction reverses.
Key findings:
- Representative embedding and late-interaction retrievers score high on the cue-available controls but consistently low on Relational Flip; the ordering persists in an LLM-assisted, manually reviewed realistic variant
- An exact margin decomposition shows both directions are correct only when direction-sensitive query–document interaction exceeds the direction-invariant pair preference; such aligned interactions are usually too weak
- Diagnostic fine-tuning shows models can learn relational behavior that generalizes across relation families, with stronger generalization in larger models
More from Research
- MIT's Markus Buehler Launches Applied AI for Materials Discovery Short Course — ProfBuehlerMIT · 2026-11-13
- From-scratch 42M-parameter model handles 7 African languages, runs offline on phones — AgencyInside407 · 2026-11-13
- Aristotle Formal Verification Proves bunnie's Security Chip Conforms to RISC-V SAIL Spec — satnam6502 · 2026-11-13
- Harvard open-sources its entire ML Systems curriculum — free, 6 pillars, hundreds of pages — mdancho84 · 2026-11-13
- 1960 Science paper famously predicted doomsday on Nov 13, 2026 — voxleone · 2026-11-13
- RAAIS 2027 set for London June 11: Odyssey world models and DeepMind open-endedness on stage — nathanbenaich · 2026-11-13