New Paper Derives Exact Stability Criterion for Outer Optimizers in Communication-Efficient Distributed Training
apsarathchandar · x · 2026-10-22
A new paper studies communication-efficient distributed training, where workers take several local steps before an outer optimizer (often with Nesterov momentum) applies their averaged update. The authors derive an exact mean-square stability criterion for this combination, answering when local-step training with momentum remains stable.
More from Research
- eLife 'Landmark' paper: RNNs reveal a mechanistic theory of planning in prefrontal cortex — ShahabBakht · 2026-10-22
- Cohere's CulturalRiddles: 28 LLMs score 79.3% on cross-cultural riddles, but just 22% in some cultures — Cohere_Labs · 2026-10-22
- Berkeley-Stanford preprint: agents that periodically forget beat ones that remember everything — ombre-ai · 2026-10-22
- DeepMind AI Agent Autonomously Discovers and Lean-Verifies Rules for 11 Unsolved Turing Machines — rohanpaul_ai · 2026-10-22
- Karpathy open-sources AutoResearch: AI agents that run the full research loop autonomously — mdancho84 · 2026-10-22
- MF² benchmark at EMNLP 2026 tests VLMs' movie comprehension with 850+ fact/fib pairs — andre_t_martins · 2026-10-22