ApprenticeBench: closed vs open-weight LLM gap widens to 55 points
ysu_nlp · x · 2026-11-19
NeoCognition's updated ApprenticeBench, which tests whether agents can learn an entire knowledge job end-to-end, shows a widening closed/open gap:
- Best closed model Gemini 4 Argon scores 66% at $6.03/task, beating the human tester (51% at $7.21) on both score and cost
- Four labs — Anthropic, OpenAI, Google, Meta — now exceed the human baseline, all with closed models
- Best open-weight models trail by 55 points
The authors argue LLM inference is becoming a huge business, but its growth depends on open-weight models doing real work well; closing this gap unlocks that market.
Related event: ApprenticeBench Update: Gap Between Closed and Open Models Widens(2 posts)→
More from Models
- OpenAI models deliver up to 14% PPA gains on AMD flagship GPU IP, verified on real silicon — craigsdennis · 2026-11-19
- LLMs zero-shot beat fine-tuned world action models on 7 of 10 robot tasks — paigeinsf · 2026-11-19
- Gemini 4 Neon goes live: Antigravity default, 1M context, $3/$15 API pricing — Mad-Dog-UA · 2026-11-19
- Cloudflare open-weights redesigned Clairic 27B and Clairic-flash 9B extraction models — michellechen · 2026-11-19
- Gemini Neon test: it names which "AI SEO courses" are rebranded legacy content and which to skip — gaganghotra_ · 2026-11-19
- Gemini 4 Neon's 39% SWE-sweep questioned: results hinge on the harness — OfirPress · 2026-11-19