Aligning image and text embeddings without paired data: Wasserstein Procrustes and rotation papers
phillip_isola · x · 2026-10-31
Phillip Isola lists relevant papers behind the "align embeddings with a rotation" finding. The key one, 'Shared Geometry As A Rosetta Stone' (arXiv:2610.09411, co-authored by Isola), shows a Wasserstein Procrustes method with coarse geometric initialization can align independently trained image and text model embeddings via a single orthogonal map without any paired examples. Geometric alignment metrics predict when this works; the method greatly outperforms baselines in the few-pair regime and can even enable text-to-image generation without pairs. Also cited: new work by Balestriero & LeCun and by Gupta & Kansal on rotating CLIP-like embeddings. Takeaway: independently trained models often share enough geometry for coarse cross-modal alignment.
More from Research
- Tversky branch inside RNN cells: half the depth matches full xLSTM with 2x faster training — stanfordnlp · 2026-10-31
- AI pushes circuit lower bound from 3.1n to 3.5n as UCLA-backed challenge aims at P vs NP — soumitrashukla9 · 2026-10-31
- Qwenputer: a full RISC-V CPU inside Qwen3.5-4B weights, running C in forward passes — armand_ruiz · 2026-10-31
- What is work for? Cross-national survey of 8,234 adults on meaning in the AI age — soumitrashukla9 · 2026-10-31
- Paper: benchmark difficulty coverage is skewed, leaderboard conclusions need caution — Miserable-Dare5090 · 2026-10-31
- AISTATS 2027 workshop proposals due Nov 6 after one-week extension — qberthet · 2026-10-31