TwinLab-Bench tests AI scientists in agentic twins of real labs; best model scores just 45.0

zju-community · hf · 2026-11-20

Zhejiang University introduces TwinLab-Bench, evaluating AI scientists on self-designed experiments across 52 tasks from real physical-science, life-science, and engineering labs, supporting 100+ experimental action types. Each task ships a TwinLab agent acting as the environment, executing the tested agent's designs against real lab records, nearly eliminating deviating readings and inconsistent repeat answers, with closed-book probes controlling for memorization. Frontier models still struggle: the strongest, Claude Fable 5.2, averages only 45.0, and the authors analyze where it fails.

Original post →

More from coding & agent

coding & agent channel →