When do separately-trained multimodal models provably converge? Good views
randall_balestr · x · 2026-10-30
Theorem statement from the thread: models trained apart on different modalities provably converge when their positive views are observations of one scene — two captions of a photo, two frames of a video, etc. With such good views, JEPAs on any number of modalities learn the same embedding up to a rotation, enabling a captioner that never saw images.
More from Research
- Steering an "Einstein neuron" in Qwen 4B produces dramatically different answers — thursdai_pod · 2026-10-31
- Protein binder scoring head-to-head: ipDAE consistently strong, OpenDDE+ipTM surprisingly good at 80% PPV — AllThingsApx · 2026-10-31
- Turing TTS benchmark: best voice AI fools listeners only ~10% of the time — omarsar0 · 2026-10-31
- Ablation study: fine-grained MoE is the best LLM architecture change since GPT-3 — stochasticchasm · 2026-10-31
- Yann LeCun to Keynote MAIN2026 on World Models for Reasoning and Planning — karimjerbineuro · 2026-10-31
- Genentech team's Nona paper sparks debate: hybrid multimodal models may beat DNA LMs as samplers — anshulkundaje · 2026-10-31