Two decades of online convex optimization: AdaGrad co-inventor Elad Hazan's memoir
机器之心 · wechat · 2026-10-18
Princeton professor Elad Hazan reflects on 20 years of online convex optimization: from Online Newton Step (rejected by FOCS/STOC) and the universal portfolios connection, to co-inventing adaptive gradient methods (AdaGrad) with Yoram Singer and John Duchi at Google, 20 papers with Satyen Kale, spectral filtering for learning dynamical systems, and its evolution toward Spectral Transformer architectures. He closes by tying to the recent Fields-medalist letter on AI in mathematics: AI excels at the theorem-proving-style skills prized by STOC/FOCS but is weaker at inventing new frameworks, so choosing interesting problems and creating frameworks becomes the mathematician's core value.
More from AGI Musings
- Super-Worms Warning: AI-Powered Self-Propagating Malware and a Fragmenting Internet — JensHonack · 2026-10-23
- Wolfram Opens Rule 30 Prizes to AIs: $30,000 for Three Open Problems — cephaloform · 2026-10-23
- Mathematicians and Biomedical Researchers React to AI in Strikingly Different Ways — manorlaboratory · 2026-10-23
- Stanford and Databricks use AI agents to reproduce medical findings and test analytical robustness — ChengleiSi · 2026-10-23
- ISMS Fellow Avi Goldfarb: AI disruption raises returns to exploring research directions — avicgoldfarb · 2026-10-23
- signull on AI-found PHP exploits: economic liability is the forcing function legacy software needs — signulll · 2026-10-23