TwinLab-Bench tests AI scientists in agentic twins of real labs; best model scores just 45.0
zju-community · hf · 2026-11-20
Zhejiang University introduces TwinLab-Bench, evaluating AI scientists on self-designed experiments across 52 tasks from real physical-science, life-science, and engineering labs, supporting 100+ experimental action types. Each task ships a TwinLab agent acting as the environment, executing the tested agent's designs against real lab records, nearly eliminating deviating readings and inconsistent repeat answers, with closed-book probes controlling for memorization. Frontier models still struggle: the strongest, Claude Fable 5.2, averages only 45.0, and the authors analyze where it fails.
More from coding & agent
- What 1.5 Years of Building Agents Taught Me: It's All a While Loop, Tools, and Specs — khant_dev · 2026-11-20
- Building an AI phone agent: 2018's duct-tape pipeline vs 2026 — JafarNajafov · 2026-11-20
- Open-Source AI Agent Eigin Will Pay Contributors for Specs, Not Bounties — danielrothmann · 2026-11-20
- X creator shares how agents now handle scheduling, research and data analysis — superinvestor_43 · 2026-11-20
- LoRA adapter lifts Qwen3.8-27B to 63% on SWE-Bench Pro, 60% fewer tool calls — ben_burtenshaw · 2026-11-20
- Mubit + Neon: agent middleware cuts frontier-model tokens by 73% — JeremyCMorgan · 2026-11-20