New paper: training tasks act as attractors, clustering network weights by dataset
wschroll · x · 2026-10-26
Researchers from FAU Erlangen-Nürnberg and collaborators published a paper in Frontiers in Cognition exploring whether different learning tasks act as attractors pulling neural networks toward distinct clusters in weight space.
- They permutation-aligned hidden units of independently initialized shallow networks to strip out equivalences before comparing weights
- Networks trained on the same dataset developed more similar weight structure than those trained on different datasets
- In the first few SGD updates, accuracy rose sharply while displacement from initialization stayed small — and these small movements were structurally meaningful, not neutral
The results support an attractor framework: even stochastic training organizes models into reproducible, task-dependent regions of state space.
More from Research
- ETH Zurich paper: dynamic evaluation allocation matches 10x-budget model rankings — zouharvi · 2026-10-26
- DuoLM: Fast LLM-Based Personalized Recommendation via Compact Encoder-Ranker — GoogleDeepMind · 2026-10-26
- DISC Learns to Combine Time-Series Dataset Similarity Metrics for Transfer Prediction — Yen-Ku Liu · 2026-10-26
- Adobe's SliderEdit++ Improves Slider-Based Image Editing by 18.5% with New SlideBench — Arman Zarei · 2026-10-26
- DeepMind Builds Auditable AI Text Detectors with Sparse Autoencoders — GoogleDeepMind · 2026-10-26
- CoRE Framework Learns Rule Routing for Knowledge Retrieval Without Supervision — Zhisheng Qi · 2026-10-26