Hyperball Optimizer Breaks Muon Scaling Bottleneck for 30% Speedup

A new paper introduces Hyperball, an optimizer wrapper that overcomes the diminishing performance gains of matrix optimizers like Muon as model scale increases. By rethinking weight decay, Hyperball achieves a 20-30% speedup in LLM pretraining.

2026-08-11 ~ 2026-08-12 · 2 related posts

• Hyperball optimizer wrapper boosts Muon pretraining speed by 20-30% — teortaxesTex · 2026-08-11
• Hyperball Optimizer Boosts Pretraining Speed by 20-30% When Combined with Muon — burny_tech · 2026-08-12