MIT Researchers Let an Agent Rewrite the Optimizer's Source Code During Training
vincesitzmann · x · 2026-10-22
Phillip Isola shared new work (SimulatedAnneal): current optimizers are static during training, so the team let an agent watch training dynamics and rewrite the optimizer's source code as training progresses — a "reactive optimizer."
Key results:
- In every language and image setting without a tuned recipe, it matches or beats the best one-shot recipe.
- The agent guards against loss spikes and overfitting on its own.
- The authors argue optimization decisions may soon belong to agents, not humans.
Paper and code are available to try on your own tasks.
More from coding & agent
- Classic 'too many tabs' meme applied to coding agents: send help — mrdrozdov · 2026-10-22
- YC F26 startup Latent reads model internals to catch hallucinations at 530x lower cost — ycombinator · 2026-10-22
- We need an L0.5 for continual learning: agent state stacks up in 4 layers — willcb · 2026-10-22
- DuckDB + Hugging Face Amsterdam workshop: build an AI feedback pipeline over 2.1M Amazon reviews — lhoestq · 2026-10-22
- Veracode report: AI now writes half of committed code but only 56% pass security checks — WeldPond · 2026-10-22
- Will Brown: Continual Learning in the Harness Is Like a Computer Without RAM — willcb · 2026-10-22