Dynamical releases SDL-2: frontier agents make correct materials-science calls in only ~half of experiments
BenBlaiszik · x · 2026-11-03
Dynamical Systems has released SDL-2, an open evaluation measuring whether AI agents can choose their own experiments to decide if a magnet-processing route will reliably meet a performance requirement.
- Setup: A digital twin of magnet processing calibrated with Rensselaer Polytechnic Institute data; agents face unfamiliar powders, uncertain physics, limited budgets and batch-to-batch variation.
- Results: Frontier models reached correct, evidence-supported decisions in roughly half of the 32 simulated investigations, pointing to reasoning under variation as the key gap.
- Thesis: The team bets on meta-learning — agents learn the relationship between evidence and action from verified experimental experience, reducing experiments needed for each new capability.
- Key distinction: One magnet meeting spec is not a reliable process; the question is whether a fresh batch made the same way will also meet it.
More from AGI Musings
- Is consciousness computable? The debate that decides immortality in 20 years — fkasummer · 2026-11-03
- Jack Clark's weekly Tech Tales: children play god in 2040s city-building game — jackclarkSF · 2026-11-03
- MATS researcher studies who captures AI's economic gains, seeking interviews — luke_drago_ · 2026-11-03
- Researcher: I want one novel discovery to my name before AI sprints past us — generativist · 2026-11-03
- AI isn't more expensive than humans: capability cost curves crash, retail prices mislead — Nawozane · 2026-11-03
- Giraffe evolution meme: AI's low-hanging fruit keeps hanging higher every year — itsOmSarraf_ · 2026-11-03