MuonH with a minus-sqrt LR schedule cuts modded-nanogpt Track 3 by 75 steps
liliang_ren · x · 2026-07-21
We submit MuonH with a minus-sqrt learning-rate schedule \((1-\sqrt{t})\) to the modded-nanogpt Track 3 benchmark.
- The new schedule improves the current MuonH SOTA by 75 steps.
- It also nearly doubles the peak learning rate under this setup.
- The comparison chart shows the new curve converging faster in the tail than the prior MuonH runs.
More results are promised later.
More from Research
- Speculative Diffusion Paper Maps the Speed-Quality Pareto Frontier via Two Tradeoffs — UCLA · 2026-10-26
- Git Records Fall Short for Auditing AI's Role in Research Repositories, Paper Finds — Koutian Wu · 2026-10-26
- What a 'Fused DeepLIFT' Actually Means: Gradient Overrides Without Hooks — jmschreiber91 · 2026-10-26
- New paper: AI agents negotiate badly under uncertainty and break deals on purpose — menhguin · 2026-10-26
- Cherimoya v0.3.1 Ships Fused DeepLIFT, ~7x Faster Attribution for Genomics Models — jmschreiber91 · 2026-10-26
- Proto-tools, a universal infrastructure layer for generative biology, now open source — jmschreiber91 · 2026-10-26