Mathematician Uses GPT to Solve Years-Old Gradient Descent Convergence Problem
konstmish · x · 2026-08-03
The author and a collaborator wrote a 2019 paper on adaptively estimating the gradient Lipschitz constant in gradient descent. In 2023, they tightened the constant factor in their estimate from 1/2 to 1/√2, but couldn't determine if it could be tightened further to exactly 1.
GPT finally provided the answer: even with a constant factor of 0.98, the method fails to converge. The model generated a PEP-style (Performance Estimation Problem) counterexample, which the author had failed to find manually back in 2019. The author notes this is a great example of how LLMs are highly useful for doing mathematics.
Related event: Researcher Uses GPT to Solve Long-Standing Gradient Descent Proof(2 posts)→
More from Research
- Applying Mechanistic Interpretability to Video Models: Exploring Latent Space Physics Simulators — mathemagic1an · 2026-08-03
- NBER Paper: Automation Strips Work of Meaning, Potentially Harming Workers — joshgans · 2026-08-03
- Matrix Factorization Approach for Dynamic Rank/Select in Data Streams — minilek · 2026-08-03
- Explorative Modeling: A Third Pretraining Axis Beyond Parameters and Data — PMinervini · 2026-08-03
- HOST: Open-Source Robot Learns New Skills in 29s from Single Video, 507x Faster Than SFT — chris_j_paxton · 2026-08-03
- Transformers Effectively Compute Amortized Gradients of a Non-Conservative Field — teortaxesTex · 2026-08-03