Fractal basins trap latent reasoning: chaos explains why LLMs overthink hard problems
wgilpin0 · x · 2026-09-09
William Gilpin's group posted an arXiv paper, Fractal basins trap latent reasoning, using nonlinear dynamics to explain why reasoning models slow down on hard tasks.
Key findings
- Leading reasoning models behave as dynamical systems with fractal basins, showing transient chaos on Sudoku, mazes, visual puzzles, math logic and even ARC-AGI; fractality increases with task difficulty.
- Reasoning slowdowns come from trajectories getting trapped near saddle points, which correspond to nearly-correct attempted solutions — the model wanders among plausible answers.
- This explains 'overthinking': similar prompts can produce reasoning traces differing 10x in length and token cost.
- The authors argue transient chaos is an inevitable tradeoff: sensitivity is needed for hard computations, but inference cost becomes unpredictable.
- A loop transformer trained on integer linear regression shows a mid-training bifurcation where fractal basins appear; the fast Lyapunov indicator correlates with how many candidate solutions the model visits before converging.
More from AGI Musings
- Anti-AI content is algorithm gold and very profitable, quips commentator — annetgriffin · 2026-10-26
- Silicon Valley breakdown: five Stanford-area giants worth ~$18T — FinanceYF5 · 2026-10-26
- Anthropic says Claude leads 26% of its AI R&D — but self-improvement still has limits — Exponential View (Azeem Azhar) · 2026-10-26
- Humanoids shattered forecasts: half-marathon record in 2026, 100m faster than Bolt — ben_j_todd · 2026-10-26
- Stephen Casper on podcast: what if AI becomes a parasitic peer species in cyberspace — StephenLCasper · 2026-10-26
- Behind the EMNLP keynote: LMs as simulated learners to generate child language acquisition hypotheses — najoungkim · 2026-10-26