Stanford Report: Open-weight models catching up, gap with closed narrows to 1.7%
mdancho84 · x · 2026-07-25
Based on the Stanford 2025 AI Index report, the author highlighted key industry trends:
- Open vs. closed tightening: Open-weight models are catching up fast. By Feb 2025, the gap between top open and closed models on Chatbot Arena narrowed to just 1.70%.
- Benchmarks crushed quickly: Scores on hard benchmarks like MMMU, GPQA, and SWE-bench jumped by 19 to 67 percentage points from 2023 to 2024, leading to a benchmark exhaustion problem.
Related event: Stanford 457-Page AI Index Report: Costs Drop, Open-Source Closes Gap(6 posts)→
More from AGI Musings
- "ChatGPT 6 Makes Workers with IQ Below 130 Useless": French AI Debate Sparks Backlash — mitchdeg · 2026-09-11
- 'AGI is here' vs reality: AI labs still ship some of the jankiest desktop apps ever — MilesCranmer · 2026-09-11
- Harry Collins: LLMs can't do frontier science because they can't invent new language — whoamisri · 2026-09-11
- The Waymo effect: how AI is quietly making research less collaborative — JohnHammersley · 2026-09-11
- Misquoted: Anthropic Staff Warned of Double-Digit Extinction Risk by 2030, Not Dismissed It — davidmanheim · 2026-09-11
- Economist Ben Moll: You Can Model Anthropic's 15% AI GDP Growth, But It Won't Happen — sebkrier · 2026-09-11