Anchor Divergence paper: contextual semantic similarity on fixed embeddings

victorveitch · x · 2026-10-19

A new arXiv paper (2610.06919) by Akash Kannan, Kiho Park, and Victor Veitch tackles the context-dependence of semantic similarity. Key points:

Semantic similarity is inherently contextual — two images may be similar by object, style, or clinical finding — yet cosine similarity fixes a single geometry
The paper shows contrastive representations naturally encompass a family of geometries, via a correspondence between probability distributions over "anchors" and Bregman geometries on the representation space (linking contrastive learning, exponential families, and information geometry)
This yields "Anchor Divergences": specifying context-specific semantic similarity on fixed representations — modeling the anchor distribution models the geometry itself
Retrieval experiments show it's an effective, efficient way to get context-specific neighborhoods (e.g., retrieve by breed, background, or clothing from the same query)

Code is available.

More from Research