Fractal basins trap latent reasoning: transient chaos explains overthinking

On September 9, William Gilpin's team published an arXiv paper, "Fractal basins trap latent reasoning" (6 pages, with code), using nonlinear dynamics to explain why reasoning models "think too long" on hard problems: reasoning trajectories get trapped in fractal basin structures exhibiting transient chaos. Two similar prompts can produce reasoning trajectories differing 10x in length, directly inflating token costs. The authors conjecture that transient chaos may be an unavoidable cost of training models to solve hard problems.

Confirmed

• Method: parameterize "problem + model weights" as a dynamical system with the solution as a fixed point; repeatedly solve the same problem, perturbing only the initial hidden state along random 2D slices. Coloring initial conditions by convergence time reveals clear fractal patterns, observed across all tested architectures and tasks including math, Sudoku, mazes, ARC-AGI with recurrent-depth models, and mainstream reasoning models.
• Mechanism: when reasoning stalls near wrong answers, the hidden dynamics encode them as saddle points. The authors liken the high-dimensional dynamics to a Plinko board—trajectories from nearby starting points converge to the same answer but scatter differently at saddle points ("saddle-point scattering"), letting the model examine more candidate solutions. PCA visualization shows adjacent initial conditions where one solves quickly while another lingers near wrong answers. The fractals arise from transient chaos: neighboring initial conditions reach the same equilibrium via wildly different paths, akin to turbulence and biological networks.
• Quantification: fast Lyapunov indicators measure transient chaos intensity and correlate with reasoning stalls.
• Difficulty effect: harder problems produce stronger fractal basin structure; the authors view fractal basins as a general physical effect when models map hard problems to analogous systems.
• Training dynamics: training a loop transformer on integer linear regression shows a mid-training bifurcation where the model suddenly generalizes to unseen samples while fractal basins emerge in the loss landscape, suggesting fractal structure and generalization emerge in lockstep.
• Reception: researchers including Stefania Druga, burnytech, and Ayush Thakur shared the work as a new lens on reasoning models' internal computation.

Unconfirmed

• The claim that transient chaos is an unavoidable consequence of training on hard problems remains a conjecture pending further experiments.

Why it matters

• This is the first nonlinear-dynamics account of reasoning-model overthinking, providing new tools (fractal visualization, Lyapunov indicators) for inspecting model reasoning.
• Up to 10x variance in reasoning length across similar prompts implies unpredictable order-of-magnitude token cost swings, directly relevant to reasoning-service cost estimation and deployment.
• The bifurcation experiment links the emergence of generalization with chaotic structure, offering training-level evidence that solving hard problems may inherently entail chaos.

2026-09-09 ~ 2026-09-10 · 16 related posts

• [source] Fractal basins trap latent reasoning: chaos explains why LLMs overthink hard problems — wgilpin0 · 2026-09-09
• Research group finds reasoning models produce fractals when solving hard problems — wgilpin0 · 2026-09-09
• Where you start reasoning hugely affects convergence time, colored as fractals — wgilpin0 · 2026-09-09
• Harder problems strengthen fractal basins, a generic effect in reasoning models — wgilpin0 · 2026-09-09
• Fractals form via transient chaos: adjacent starts, wildly different routes — wgilpin0 · 2026-09-09
• Saddle points in latent dynamics explain why reasoning gets stuck near wrong answers — wgilpin0 · 2026-09-09
• High-dimensional reasoning dynamics behave like a Plinko game of saddles — wgilpin0 · 2026-09-09
• Saddle scrambling lets reasoning models explore more solutions, measured by Lyapunov indicator — wgilpin0 · 2026-09-09
• Mid-training bifurcation: fractal basins emerge as a transformer learns to generalize — wgilpin0 · 2026-09-09
• Reasoning models 'overthink': similar prompts yield 10x difference in reasoning length and cost — wgilpin0 · 2026-09-09
• [source] Transient chaos may be an inevitable cost of training models to solve hard problems — wgilpin0 · 2026-09-09
• Loop transformer hits mid-training bifurcation as fractal basins emerge alongside generalization — Stefania_druga · 2026-09-10

4 near-duplicate retellings: wgilpin0 · ayushthakur0 · burny_tech · Stefania_druga