Fractal basins and transient chaos explain why reasoning models overthink

William Gilpin's team published a paper, "Fractal basins trap latent reasoning" (6 pages, code included), on arXiv on September 9, using nonlinear dynamics to explain why reasoning models "think too long" on hard problems. The conclusion: overthinking is not accidental but arises from transient chaos and fractal basin structures.

Confirmed

• Method: they parameterize "problem + model weights" as a dynamical system where solutions correspond to fixed points; repeatedly solving the same problem while perturbing the initial hidden state only on random 2D slices reveals that reasoning models literally "draw" fractals while solving.
• Experiments cover recurrent depth models and mainstream reasoning models on math, Sudoku, maze, and other tasks.
• Core finding: reasoning trajectory lengths for similar prompts can differ by a factor of 10, directly inflating token costs and making inference cost hard to predict; "thinking longer" on hard problems manifests as transient chaos.
• The authors conjecture that transient chaos is unavoidable when training models on hard problems, and may be an inevitable price of the ability to solve them.

Why it matters

• This is the first study to characterize reasoning models' overthinking from a nonlinear dynamics perspective, offering a new tool (fractal structure visualization) for observing model thinking.
• The results show order-of-magnitude, unpredictable swings in token costs, directly relevant to cost estimation and deployment of reasoning services; if transient chaos is indeed the price of training on hard problems, a capability-vs-cost trade-off may be needed.

2026-09-09 ~ 2026-09-09 · 13 related posts

• [source] Fractal basins trap latent reasoning: chaos explains why LLMs overthink hard problems — wgilpin0 · 2026-09-09
• Research group finds reasoning models produce fractals when solving hard problems — wgilpin0 · 2026-09-09
• Where you start reasoning hugely affects convergence time, colored as fractals — wgilpin0 · 2026-09-09
• Harder problems strengthen fractal basins, a generic effect in reasoning models — wgilpin0 · 2026-09-09
• Fractals form via transient chaos: adjacent starts, wildly different routes — wgilpin0 · 2026-09-09
• Saddle points in latent dynamics explain why reasoning gets stuck near wrong answers — wgilpin0 · 2026-09-09
• High-dimensional reasoning dynamics behave like a Plinko game of saddles — wgilpin0 · 2026-09-09
• Saddle scrambling lets reasoning models explore more solutions, measured by Lyapunov indicator — wgilpin0 · 2026-09-09
• Mid-training bifurcation: fractal basins emerge as a transformer learns to generalize — wgilpin0 · 2026-09-09
• Reasoning models 'overthink': similar prompts yield 10x difference in reasoning length and cost — wgilpin0 · 2026-09-09
• [source] Transient chaos may be an inevitable cost of training models to solve hard problems — wgilpin0 · 2026-09-09

2 near-duplicate retellings: wgilpin0 · ayushthakur0