Replacing Fragile Heuristics in DINOv2 with Pure Optimal Transport
RexDouglass · x · 2026-07-30
The thread discusses the technical limitations of current Self-Supervised Learning (SSL) models like DINOv2. The author notes that while these models are engineering marvels, they rely on fragile hacks such as EMA, stop-gradients, and custom centering to prevent representation collapse.
It raises the question of whether these empirical heuristics could be replaced entirely by pure, mathematically sound optimal transport theory to fundamentally improve training stability.
More from Research
- MIT's Neil Thompson: AI comes for tasks before jobs, and fixed costs favor big firms — bigdata · 2026-10-16
- UMD's PAIR framework bridges perception and action in VLA models via bridge tokens — Kaixi Feng · 2026-10-16
- CI-MSE offline metric hits -0.87 Spearman correlation with real robot task success vs weak raw MSE — chris_j_paxton · 2026-10-16
- Colossal Biosciences uses AI to decode genetic switches behind the extinct dodo — castrotech · 2026-10-16
- Colossal Biosciences uses AI to identify gene switches in dodo de-extinction push — castrotech · 2026-10-16
- Cheng Lou open-sources Handwired: hand-written weights turn code into training-free neural nets — dotey · 2026-10-16