Moving ViT weights 1.3% without changing predictions scrambles representations, challenging Platonic hypothesis
neuroecology · x · 2026-10-15
A new paper introduces Hessian Null Space Continuation (HNC), a method for wandering through the dense valley of equally good neural network solutions, and it challenges the Platonic Representation Hypothesis.
- Key result: shifting a ViT's weights by just 1.3% along the Hessian null space — with predictions fully preserved — makes its representations less similar to the original than a random-init ViT is, showing vast representational diversity around any trained network.
- Some solutions found via HNC remain representationally aligned with other models or brains, others not at all, suggesting alignment is a consequence of optimizer choices rather than task optimization alone.
- Optimistic angle: HNC can steer toward solutions with desirable properties (sparsity, safety) without sacrificing performance.
- Work led by Ann Huang; authors stress alignment isn't "fake", just not inevitable.
Related event: HNC Method Challenges Platonic Representation Hypothesis(3 posts)→
More from Research
- RAI Institute's single-track bicycle robot UMV lands in Science Robotics — joemeno · 2026-10-15
- Five-year Harvard postdoc preprint unveils BARRACUDA for systematic synthetic circuit design — NikoMcCarty · 2026-10-15
- Autonomous LLM agents for materials science: seminar at Texas Southern University — Fowe · 2026-10-15
- Behavioral Self-Rewarding lifts Qwen3-14B to 34.7% on SWE-bench Verified — dair_ai · 2026-10-15
- vLLM adaptive verification now works head-free, using draft logits with online logistic calibration — vllm_project · 2026-10-15
- New arXiv paper: thousands of ML models poorly predict hundreds of life outcomes — RobbWiller · 2026-10-15