Viewing LLMs as Learned Optimizers: A Multi-Start Optimization Perspective
YouJiacheng · x · 2026-08-04
The author shares a classic reference on multi-start methods and proposes a new perspective for model optimization (related to the XM paper):
- Models can be treated as learned optimizers.
- Using restart or multi-start strategies is common practice in non-convex optimization.
- Therefore, we can train a learned optimizer using a multi-start approach.
- However, a potential challenge is the train-test mismatch this might introduce.
Related event: New Perspective on Model Training: Multi-Start Restart Strategies(3 posts)→
More from Research
- LeRobot now supports 30+ robot hardware integrations with drop-in plugins — m_wulfmeier · 2026-08-04
- Roasting Academia: NeurIPS is Essentially Just Scrolling OpenReview — abursuc · 2026-08-04
- Open-Source LoRA for Satellite Image Editing Uses AI to Auto-Generate Training Data — LimitlessSaint · 2026-08-04
- Weekend Project: RL-Trained 4B LLM Rewrites AI Text to Fool Open-Source Detectors — matthen2 · 2026-08-04
- NUS Creates Octopus-Inspired Swimming Robot Driven by Just Two Motors — lukas_m_ziegler · 2026-08-04
- Native Tool Calling and Correct Sampling Params Boost LLM Evals by >30 Points — xeophon · 2026-08-04