A new notebook shows learning-rate boundaries can be fractal
S_Conradi · x · 2026-07-23
Hyperparameter tuning is described as literally fractal: the boundary between learning rates that train and those that diverge behaves like a coastline with infinitely detailed inlets.
The post links this behavior to gradient descent as an iterated map, compares it to the Mandelbrot set, and says an interactive notebook can recompute 65,536 full training runs on a GPU in about one second while zooming into the boundary. The same fractal pattern also appears on a real dataset (MNIST-1D), and the notebook was built with marimo, JAX, marimo-pair, and Claude Code.
Related event: Neural Network Hyperparameter Boundaries Found to be Fractal(4 posts)→
More from Research
- AutoCAD-Bench puts GPT 5.6 sol at 46% on precise CAD tasks via computer use — DevvMandal · 2026-07-24
- Convokebio releases a free drug-pipeline database built for humans and LLMs — akilian · 2026-07-24
- Research paper count should not be a proxy for quality, says a post praising Imperial's top-3-paper review — ZeeshanZiaML · 2026-07-24
- RoboPapers previews a HUG episode on human universal grasping — chris_j_paxton · 2026-07-24
- Photon-1 learns computer use from 18 years of unlabeled screen recordings — ycombinator · 2026-07-24
- GoodfireAI validates gene-variant predictions against biobank, clinical, and RNA data — leedsharkey · 2026-07-24