Researchers find edge of stability for Muon and all non-adaptive optimizers, cut nanoGPT record by 3.2%
andersonbcdefg · x · 2026-11-06
Researchers report discovering the edge of stability for Muon — and for every other non-adaptive optimizer — naming the phenomenon "bubbleforce". Building on this insight they shaved 100 iterations off the nanoGPT Muon record, a -3.2% improvement, suggesting a better theoretical understanding of optimizer stability translates directly into faster training.
More from Research
- New SSRN paper examines data center siting and the rising wave of community opposition — HooverInstitution · 2026-11-06
- Caltech's Van Valen Lab ditches X, publishes all AI models on Hugging Face — anshulkundaje · 2026-11-06
- Soumith Chintala: Inkling 2 base model beats GLM-5.3, DeepSeek V4 on code — soumithchintala · 2026-11-06
- Cartesia's H-Net shows in-context tokenization: models can learn to chunk data without a tokenizer — TheTuringPost · 2026-11-06
- Independent audit: RoundSquare's zero-axiom Lean 4 proof compiles cleanly but doesn't resolve Clay Yang–Mills — kevrussell · 2026-11-06
- Learning-theoretic protocol lets two companies privately test data compatibility before pooling — roydanroy · 2026-11-06