NeurIPS paper: parallel-in-time RNN training with DEER+GTF speeds up chaotic-system training over 100x
DangerousFunny1371 · reddit · 2026-10-01
A NeurIPS 2026 spotlight paper presents a parallel-in-time training method for nonlinear RNNs, achieving >100x speedup on dynamical systems reconstruction (DSR) with very long chaotic time series (T>10^6):
- DEER solves the RNN forward pass via Newton-type fixed-point iterations across the whole sequence, scaling as O[(log T)²] instead of O[T] for GPU parallelization — but it breaks down under chaotic dynamics, degrading to O[T log T].
- Generalized teacher forcing (GTF) stabilizes DEER against chaotic divergence and reduces exposure bias versus standard teacher forcing.
- The combination hugely outperforms Mamba and other state space models in the DSR setting.
Preprint: arxiv.org/abs/2605.12683
More from Research
- Pleias releases TD Bench, a controlled multilingual benchmark of fluency judge models across 25 languages — antoine_chaffin · 2026-10-23
- AI suddenly solves 80-year-old math problems: humans search breadth-first, LLMs go depth-first — Latent Space · 2026-10-23
- Shared-control maze study shows stronger LLM agents involve humans only when it pays off — 5a-community · 2026-10-23
- OpenClaw finds and fixes a dozen bugs in latest Gaussian16 C.01 quantum chemistry code — heyneighbor · 2026-10-23
- Claude Science helped trace the DR Congo Ebola outbreak from 838 virus genomes — nc_frey · 2026-10-23
- 20+ ICLR 2027 submissions caught with AI-generated placeholder results left in captions — thegautamkamath · 2026-10-23