Functional Gradient Descent with Adaptive Representations accepted at NeurIPS, beats neural nets by ~10x
dccsillag0 · reddit · 2026-09-28
The authors' NeurIPS-accepted paper formalizes a broad class of approximation schemes ("adaptive representations") for functional gradient descent, provably ensuring convergence to the global minimizer while remaining directly implementable.
- Problem: Functional GD often outperforms neural nets, but functional gradients are infinite-dimensional and naive approximations converge to the wrong place.
- Result: The resulting algorithms often beat corresponding neural networks by an order of magnitude across multiple settings.
Paper: arxiv.org/abs/2606.16926. First author is answering questions in the thread.
More from Research
- Stanford lands $25M NIH grant to build AI center for personalized dementia care — StanfordAILab · 2026-09-29
- RL Gains May Hit a Wall: Verifiable Data Limits and Stalling nanochart Benchmarks — QuintinPope5 · 2026-09-29
- TALES Benchmark on LLM Game-Playing Accepted to NeurIPS 2026, New Results Coming — tw_killian · 2026-09-29
- UBC's Schmidt wins ECML PKDD Test of Time Award for 2016 gradient convergence paper cited 2,000+ times — MarkSchmidtUBC · 2026-09-29
- Amazon AGI's AutoGym auto-generates tasks, environments, and verifiers for agent RL training — omarsar0 · 2026-09-29
- RECLAIM preprint: best AI agent reproduces only 41% of papers with code, 15% without — VraserX · 2026-09-29