Phillip Isola demos aligning image and text embeddings with just a single rotation
phillip_isola · x · 2026-10-31
MIT professor Phillip Isola shares a demo by Randall Balestriero and Yann LeCun showing that image and text embeddings can be roughly aligned with just a rotation. In the demo, frozen DINOv2/DINOv3 vision embeddings are mapped via one orthonormal linear map (fitted on 230,014 image-text pairs) into a text-only JEPA space, then decoded by a language model. Testing it on his cat Moxie's photo, Isola reports the caption was completely accurate — supporting the Platonic Representation Hypothesis that independently trained models converge toward shared geometry.
More from Research
- Tversky branch inside RNN cells: half the depth matches full xLSTM with 2x faster training — stanfordnlp · 2026-10-31
- AI pushes circuit lower bound from 3.1n to 3.5n as UCLA-backed challenge aims at P vs NP — soumitrashukla9 · 2026-10-31
- Qwenputer: a full RISC-V CPU inside Qwen3.5-4B weights, running C in forward passes — armand_ruiz · 2026-10-31
- What is work for? Cross-national survey of 8,234 adults on meaning in the AI age — soumitrashukla9 · 2026-10-31
- Paper: benchmark difficulty coverage is skewed, leaderboard conclusions need caution — Miserable-Dare5090 · 2026-10-31
- AISTATS 2027 workshop proposals due Nov 6 after one-week extension — qberthet · 2026-10-31