OPAL optimizer auto-detects and suppresses loss spikes, beating Muon and AdamW under data corruption
vincesitzmann · x · 2026-10-22
Vince Sitzmann demos the OPAL optimizer noticing loss spikes, zeroing them out, and guarding against them afterwards—even rolling back steps and resetting momentum when it overshoots a minimum. On ResNet-18/CIFAR-10 with four injected random-data corruptions, OPAL's test-accuracy deltas (-50.0, -66.3, -12.1, +0.7) beat Muon+clip, Muon, AdamW variants, and Stable-SPAM, showing reactive agents can learn to defend corruptions after a few exposures.
Related event: EPFL's OPAL Puts a Coding Agent Inside the Training Loop(8 posts)→
More from Research
- Cohere Labs teases CulturalRiddles, a benchmark testing AI's grasp of cultural context — Cohere_Labs · 2026-10-22
- Microsoft Research claims proof of the 31-year-old KLS conjecture, aided by human-directed AI — LesterMackey · 2026-10-22
- SGLang ran early RL training on TPUs with Miles and TorchTitan, with promising results — ying11231 · 2026-10-22
- Rohan Paul shares full technical report of Recursive Flint — rohanpaul_ai · 2026-10-22
- Prompt Injection as Role Confusion: 60% attack success on frontier models, ICML 2026 paper — ChrisGPotts · 2026-10-22
- Benchling Post-Trains NVIDIA Nemotron 3.5 Super VL, Beating Closed Models on Lab Tasks at Low Cost — NVIDIAAI · 2026-10-22