DINOv3 patches encode position so strongly by default that you need to clean them up
StephanSturges · x · 2026-11-22
The author observes that DINOv3's patch features encode position to a staggering degree by default—likely inflating benchmark performance on some paper tasks but problematic for real-world usage. Accompanying PCA visualizations of aerial imagery show position gradients dominating the feature maps. The takeaway: clean the patches up before deploying them downstream.
Related event: DINOv3 patch features encode strong positional information(2 posts)→
More from Models
- Tracking how Claude's utopian visions have evolved over time — amplifiedamp · 2026-11-22
- LLMs have always self-corrected: Opus 5.5 caught its own mistake mid-generation in real time — amplifiedamp · 2026-11-22
- Open-weight models can no longer be suppressed: Pirate Face turns model weights into immortal torrents — esrtweet · 2026-11-22
- 'Compression of the RH proof has begun': AI circles buzz over unverified Riemann claim — teortaxesTex · 2026-11-22
- GLM 5.3 and DeepSeek V4.1 Flash resold at 60% off: the economics of open-source giveaways — lxfater · 2026-11-22
- GPT-6.2 Astra and Sol answer more directly but score lower on transparency than Claude — The Decoder · 2026-11-22