New Paper Unifies Design of Learning-Rate-Free Optimization Algorithms
bremen79 · x · 2026-10-30
- A new paper (arXiv:2610.39531) with Julien Grand-Clément and Joon Kwon presents a unified way to design learning-rate-free algorithms, covering both improved AdaGrad-like methods and parameter-free ones.
- The basic idea goes back to Blackwell; the key technique appeared in Grand-Clément & Kroer (NeurIPS'21, 2024). This work greatly extends it while simplifying the math.
- The write-up includes a conic FTRL theorem: with β≥2 and at=βG²t in a normed or Hilbert space, all updates are well-defined and regret is simultaneously bounded for all comparators, with RegretT(a)<ε; convergence guarantees follow by averaging iterates via Jensen's inequality.
More from Research
- Two-stage 3D ResNet pipeline detects neonatal brain pathologies from MRI with 0.919 AUC — Dr_Alex_Crimi · 2026-10-30
- 40 years of putting new neurons into the CNS: why so slow, and can AI accelerate it? — patrickmineault · 2026-10-30
- SCAMPER demo: virtual rat trots and gallops with gaits it never saw in training — tyrell_turing · 2026-10-30
- Ben Recht on efficient algorithms, the ETH, and how complexity theory lost the plot — beenwrekt · 2026-10-30
- Lambda Surgical launches first-in-human study of LiDAR-based LambdaGuide at PINE — ditzikow · 2026-10-30
- Tessl framework: right skill lifts agent accuracy 20%, but activation drops to 40% — bibryam · 2026-10-30