Autoregressive Transformers Extrapolate Chaos From Local Trajectories, Matching the Feigenbaum Constant to 5×10⁻⁴
LMU · hf · 2026-10-02
LMU researchers train small autoregressive transformers from scratch on trajectories from restricted parameter regimes of nonlinear systems (logistic/sine maps, Lorenz, generalized Hopf). Under closed-loop evaluation far outside training distributions, the models recover period-doubling cascades, chaos, and attractor structures with high fidelity—on the logistic map reproducing doublings up to period 128 with a scaling ratio of 4.6687, matching the Feigenbaum constant within 5×10⁻⁴. Causal interventions reveal how control-parameter information flows through attention into state prediction, suggesting a narrow local window suffices for global dynamical generalization.
More from Research
- Audit: 23% of "wrong" cache hits in a semantic caching benchmark were identical prompts — Reasonable_Royal_621 · 2026-10-02
- Amazon Research Awards Fall 2026 opens with 9 research tracks, deadline Nov 4 — DocXavi · 2026-10-02
- Meta research: Knowledge distillation cuts training data memorization by over 50% vs fine-tuning — lasha_nlp · 2026-10-02
- Hugging Face publishes the ultimate guide to multi-harness RL — Saboo_Shubham_ · 2026-10-02
- From Transformers to MoE: The Research Papers Behind Every Major AI Concept — goyalshaliniuk · 2026-10-02
- MoE Paper Explained: Sparsely-Gated Experts Boost Model Capacity Over 1000x — goyalshaliniuk · 2026-10-02