Simon Prince Publishes Neural ODEs Tutorial: Residual Nets as Infinite-Layer ODEs
SimonPrinceAI · x · 2026-09-28
Simon Prince has published the tenth installment of his RBC Borealis tutorial series, covering Neural ODEs.
- Construction: By sharing parameters across all layers of a residual network, the model approaches a continuous limit — infinitely many layers, each making an infinitesimal change to the representation — yielding a "neural" ODE.
- Derivations: The tutorial derives the gradient-computation algorithm for training, shows inference is equivalent to applying the Euler method to the underlying ODE, and that the gradient calculation can likewise be viewed as Euler's method on a related ODE.
- Properties: Constant memory cost during training, adaptive evaluation strategies that trade numerical precision for speed per input, and applicability to irregularly sampled time-series inference.
- Relevance: Neural ODEs underpin generative models such as flow-matching. The write-up uses matrix calculus notation throughout.
More from Research
- Mila hosts EcoHull hackathon Oct 26-27: ML to predict ship hull fouling and cut maritime carbon — Mila_Quebec · 2026-09-28
- A 24-line Python agent that pulls and ranks weekly arXiv papers in 10s — omarsar0 · 2026-09-28
- SmolDataEnvs launches: 5k+ verifiable RL tasks on real Kaggle data, fully open — SergioPaniego · 2026-09-28
- Linear mode connectivity appears on test loss but not train loss, notes researcher Arohan — _arohan_ · 2026-09-28
- John Langford ships polished Vowpal Wabbit 9.11.9, fixing the broken Maven Central release — JohnCLangford · 2026-09-28
- NeurIPS papers: CoT faithfulness metrics are near-random, plus MoE router geometry and LLM belief studies — megamor2 · 2026-09-28