Who First Saw Linear Mode Connectivity? Vaishnavh's Test-Loss Observation Predates the 2019 Paper

roydanroy · x · 2026-09-29

Roydanroy clarifies the history of linear mode connectivity (LMC): the first observation was by @vaishnavh, on test loss when swapping training data—predating Frankle and Dziugaite et al. 2019, which coined the term and defined LMC on train loss (IIUC), though his team also observed it on test and believes it appeared in the abstract.

He calls LMC one of the most underrated features of training neural networks, noting lots of cool follow-up work since.

Related event: Linear Mode Connectivity Sparks Debate Over Its Origins(2 posts)→

Original post →

More from Research

Research channel →