The open vs closed AI gap on biology is ~7 months, but the estimate is highly uncertain
Manderljung · x · 2026-10-21
GovAI researcher Hanna Palya's analysis estimates the gap between open-weight and closed general-purpose AI models on biological capabilities:
- Key finding: the best open-weight model, GLM-5.2 (weights released June 2026), trails the best closed models by about seven months on third-party biology benchmarks — the first open-weight model to match Gemini 3 Pro (Nov 2025). Since 2025 the gap has fluctuated between 7-14 months with no clear trend.
- Structure: the gap appears larger on biological knowledge benchmarks than agentic ones, but too few open-weight models have agentic results to confirm.
- Data limitations: few biology benchmarks, inconsistent evaluation conditions, and heavy refusals by closed models on high-risk benchmarks push the 95% interval from under 1 month to 14; small method changes shift the estimate between 7 and 13 months.
- Policy relevance: the gap size matters for assessing costs and benefits of frontier model safeguards, which are currently strict.
More from Safety
- Two PHP CVEs incoming, warning that '70.2% of the internet' will be scrambling — JiliJeanlouis · 2026-10-21
- Nintendo strikes 9 AI-modded game videos, including a 1.2M-view Pokémon Snap — AIandDesign · 2026-10-21
- TiDB leak report: root-level org takeover bug left unfixed for 4 years, creds in Jira — arpit_bhayani · 2026-10-21
- User tricks Gemini into confirming it's barred from answering any election questions — CapeCodder101 · 2026-10-21
- EU's multi-stakeholder working group vs America's 'let's ship it' meme — alexmacgregor__ · 2026-10-21
- Dario Amodei responds to AI safety critique: admits some mistakes, defends Anthropic's theory of change — DarioAmodei · 2026-10-21