Berkeley & Princeton release MLS-Bench to test if AI can invent new ML methods
jiqizhixin · x · 2026-08-20
UC Berkeley, Princeton, Tsinghua, and UW introduced MLS-Bench, a benchmark designed to evaluate whether AI systems can invent new machine learning methods rather than just applying existing ones.
- Scope: Covers 140 tasks across 12 domains, ranging from algorithm design to optimization.
- Mechanism: Requires agents to improve specific ML components and prove effectiveness across diverse settings and scales, preventing shortcuts and overfitting.
- Key Finding: Current agents still lag behind human-designed methods. They are decent at engineering-style tuning but fail at genuine method invention.
- Bottleneck: Throwing more compute, search, or context at the problem does not fix it; the real bottleneck is scientific insight: planning, validating, and scaling claims.
More from Research
- Reflection on RL: Good for boundaries, bad for long-term goals — sethlazar · 2026-08-20
- Rebranding STS work as technical AI safety for funding climate — evijit · 2026-08-20
- PySR v2.0 Released: High-Performance Symbolic Regression for Discovering Math Laws — MilesCranmer · 2026-08-20
- Visualizing Two-Tower MLP: Core Architecture from RecSys to CLIP — ProfTomYeh · 2026-08-20
- YC Paper Club: Deep Dive into Data, Benchmarks, and Multilingual Models — Y Combinator · 2026-08-20
- LightOn releases mLateOn: SOTA multilingual retrieval with just 115M parameters — antoine_chaffin · 2026-08-20