Neural network trainability boundary is fractal, shows Jascha Sohl-Dickstein
alec_helbling · x · 2026-09-21
Jascha Sohl-Dickstein's blog post (and short paper, arXiv:2402.06184) shows the boundary between hyperparameters where neural network training succeeds or fails has a fractal structure.
- Insight: training and fractal generation both repeatedly apply a function to its own output, with hyperparameters deciding divergence vs. boundedness, like the Mandelbrot set.
- Scanning hyperparameter space reveals gorgeous, organic fractal patterns, accompanied by mesmerizing visualizations and deep zoom videos.
- Motivated by teaching his five-year-old daughter about fractals.
More from Research
- Terry Tao's Crowdsourced Optimization Constants Repo Already Includes Komlós Conjecture as C_24 — IgorCarron · 2026-09-21
- Hugging Face Ships tokenizers v1, Often Tens of Times Faster Than v0.23 — art_zucker · 2026-09-21
- GLiNER2's schema-conditioned encoders reignite NER vs LLM extraction debate — SGmoze · 2026-09-21
- Question-first prompting took local Qwen from 89% to 100% and cut latency 5x — Micha0827 · 2026-09-21
- Shutting-down Eidon AI Open-Sources 9TB of Egocentric Housework Video on Hugging Face — victormustar · 2026-09-21
- New theory work proves quadratic separation between uniform/Gaussian diffusion and unmasking for parallel generation — LucaAmb · 2026-09-21