New training losses make ViTs develop specialized spatial attention heads
Yuji Nozawa · hf · 2026-11-10
Kioxia researchers introduce two visual focusing losses that align a Vision Transformer's multi-head attention with image regions without labels, yielding specialized spatial attention heads and improving dense prediction tasks like zero-shot segmentation.
More from Research
- Interpretability will never be solved as alien AI architectures keep proliferating — burny_tech · 2026-11-10
- New Preprint Synthesizes Symbolic Models of Human and Machine Reasoning on ARC at Scale — burny_tech · 2026-11-10
- New Interpretability Techniques Teased, Researcher 'More Optimistic Than Ever' — burny_tech · 2026-11-10
- We tried predicting the dollar with LLM news analysis — and failed — zit-hb · 2026-11-10
- Spectral vs RGB rendering: why screen-space GI spills red light unrealistically — ssh4net · 2026-11-10
- UMD team wins EMNLP 2026 Best Paper for comparing human and LLM answering strategies — boydgraber · 2026-11-10