Moving ViT Weights by Just 1.3% Leaves Predictions Intact but Representations Unrecognizable
najoungkim · x · 2026-10-15
Researcher Ann Huang presented a new paper introducing Hessian Null Space Continuation (HNC), probing how much representational diversity exists around a trained network.
Key finding: ViT weights can be moved by just 1.3% while preserving the model's predictions, yet the resulting internal representations become less similar to the original than those of a randomly initialized ViT. This reveals substantial redundant degrees of freedom inside trained networks — vastly more behavioral-preserving representational diversity than expected.
Related event: HNC Method Challenges Platonic Representation Hypothesis(3 posts)→
More from Research
- Curated roundup: frontier models help crack decades-old problems across disciplines — IgorCarron · 2026-10-15
- Stanford's CausalSmith automates causal inference research with Lean-verified proofs — StanfordAILab · 2026-10-15
- Cellular Intelligence wins Best Poster at scverse 2026 for perturbation platform — arjunrajlab · 2026-10-15
- BARRACUDA microfluidic chip traps a million single cells to map gene circuit design space at scale — NikoMcCarty · 2026-10-15
- Stanford's DexAgent turns one human egocentric video into dexterous robot training data — StanfordAILab · 2026-10-15
- Johns Hopkins wins HCOMP 2026 Best Paper for reader-centered summarization evaluation — mdredze · 2026-10-15