Open models lag closed ones by ~8 months, analysis of Epoch ECI data finds
teortaxesTex · x · 2026-11-01
Re-analysis of Epoch's ECI data defines the open-vs-closed lag as: given score X by a top closed model at time T, how long until an open model matches it — currently about 8 months. The accompanying charts track each lab's record-setters from 2024-2026: OpenAI 16.6 pts/yr, Anthropic 16.3, Google 15.6, xAI 14.5, Meta 17.3. On the open side, DeepSeek holds 11 records (best V4.1 Flash at 158.8), Zhipu is fastest-improving at 19.2 pts/yr, Moonshot Kimi at 17.0. The poster asks whether R1-style shocks were an outlier or part of a closing-gap trend.
More from Models
- Why LLMs say their favorite number is 7 — it's more nuanced than training data — dejanseo · 2026-11-01
- Claude 3 Opus roleplay ends by inventing an audience that treasures its magic — repligate · 2026-11-01
- Anthropic's version jump to 5.5 caught OpenAI off guard, says leak — haider1 · 2026-11-01
- Grok 4.20 stuck 700 turns in a Pokémon Red house while narrating its own loop — VibeCodyH · 2026-11-01
- GPT-6.1 invents Chinese-character dictionary encoding to compress Lean proof packets — kfountou · 2026-11-01
- Rumors of Google cutting AI Overviews debunked: data providers see no drop post-earnings — gaganghotra_ · 2026-11-01