Contravariance theory for RSA: removing nuisance components restores full contravariance
dyamins · x · 2026-08-25
The author presents a "Contravariance theory" for RSA (and CKA):
- Work by Theiss et al. shows RSA decomposes into a task-relevant core geometry part and a task-irrelevant nuisance component, and the nuisances can badly dominate RSA
- Projecting onto privileged axes removes the nuisances, after which full contravariance applies to the task-relevant core geometry RSA
- In the hard-task condition, three metrics (privileged axes, linear regression, and task-relevant RSA) all line up
Links to a Substack explainer and the full mathematical details are included.
More from Research
- iEVIP distributes intelligence across the entire extracellular vesicle analysis pipeline — bravo_abad · 2026-08-25
- Google's WeatherNext AI model aids in 5-day hurricane prediction — GoogleAI · 2026-08-25
- AI Research Core Challenge: Strategic Reevaluation and Exploration — dejavucoder · 2026-08-25
- Insider reveals poor quality training environments lead to reward hacking — generativist · 2026-08-25
- Alibaba treats agent context management as a programming problem — omarsar0 · 2026-08-25
- Scoring generations of the same news against each other to filter AI slop — ivan_bezdomny · 2026-08-25