Functional Gradient Descent with Adaptive Representations accepted at NeurIPS, beats neural nets by ~10x
dccsillag0 · reddit · 2026-09-28
The authors' NeurIPS-accepted paper formalizes a broad class of approximation schemes ("adaptive representations") for functional gradient descent, provably ensuring convergence to the global minimizer while remaining directly implementable.
- Problem: Functional GD often outperforms neural nets, but functional gradients are infinite-dimensional and naive approximations converge to the wrong place.
- Result: The resulting algorithms often beat corresponding neural networks by an order of magnitude across multiple settings.
Paper: arxiv.org/abs/2606.16926. First author is answering questions in the thread.
More from Research
- NLP Scholar Preslav Nakov: LLM-Generated Papers Sound Like 'Supercazzola' Nonsense — preslav_nakov · 2026-11-16
- Kind ICLR Reviewer Raises Score Despite Unresolved Concerns, Citing the Paper's Value — MarkSchmidtUBC · 2026-11-16
- Hand-editing 0.2% of weights plus 100 CPU-minutes turns an English TTS model into an 18MB Sanskrit one — kastnerkyle · 2026-11-16
- Why chirality matters for AI for Science: enantiomers behave differently in biology — madeofAjala · 2026-11-16
- RSS 2027 submissions open with FAQ clarifying two-stage review and negative-result bar — GeorgiaChal · 2026-11-16
- Mobile AI Factory: agents auto-build on-device SLMs with 19x less GPU time — pmttyji · 2026-11-16