Clock or Pizza: tiny hyperparameter changes make nets learn different algorithms
burny_tech · x · 2026-10-27
A NeurIPS 2023 paper (Zhong, Liu, Tegmark, Andreas) shows neural nets trained on modular addition do not reliably rediscover the same known algorithm.
- Small changes to hyperparameters and initialization induce qualitatively different algorithms from a fixed training set
- Some networks implement the familiar Clock algorithm; others implement a previously undescribed but comprehensible "Pizza" algorithm, or parallel implementations of multiple algorithms
- Even simple learning problems admit surprising solution diversity, motivating new tools to characterize networks across their algorithmic phase space
Related event: NeurIPS Paper: Modular Addition Networks Can Discover Novel Pizza Algorithm(2 posts)→
More from Research
- UC Berkeley paper: RL reweights LLM reasoning strategies rather than expanding them — burny_tech · 2026-10-27
- Dehaene lab finds a cortical map integrating integers and fractions via 7T fMRI — burny_tech · 2026-10-27
- LLM rewrites a physics paper into a clearer 12-page explainer — correctness unverified — thomasahle · 2026-10-27
- OpenAI's new math results accused of scooping community mathematician's Yang-Mills work — burny_tech · 2026-10-27
- NYC Reliable AI Meetup on Synthetic Data Set for Nov 16 with NYU's Gautam Kamath — karen_ullrich · 2026-10-27
- Meta's InteractiveWorld Benchmark Shows MLLMs Still Far from Reliable Physical Assistants — meta · 2026-10-27