ROR: Dynamically switch optimizers during training
RichmanRonald · x · 2026-08-20
The paper 'Many Optimizers But Only One Training Path' proposes 'Repeated Optimizer Resampling' (ROR). Unlike fixing one optimizer, ROR dynamically searches for the best one during training. Every b epochs, candidate optimizers scout for s epochs, and the best one finishes the segment. Experiments on MNIST, Fashion-MNIST, and insurance data show one-epoch ROR uses 24%-35% of the training cost of exhaustive search while matching the best fixed optimizer's performance.
More from Research
- SkillForge: Self-Distilling Agents for Project-Specific Bug Fixing — SJTU · 2026-08-20
- FM-Bench Evaluates Long-Horizon Agent Management via Football Club Simulation — Tianyou Wang · 2026-08-20
- Research: Looped Language Models Improve Compositional Tool Calling — Andrei Cristian Popescu · 2026-08-20
- Paper proposes interpretable framework for dynamic hedging using Itô Signatures — chaumian · 2026-08-20
- Paper Proposes Multi-Level Market Making Using Reinforcement Learning — chaumian · 2026-08-20
- Discussing the feasibility of representing ontologies in plain Markdown — durlabha · 2026-08-20