ApprenticeBench: closed models hit 77% vs 22% for open, widening capability wall

ysu_nlp · x · 2026-11-19

ApprenticeBench — which tests whether an agent can learn to do an entire knowledge job end-to-end through continual learning — updated its leaderboard, exposing a huge closed-vs-open gap:

ysunlp comments: for agents to take on independent knowledge work, a static harness isn't enough — every company and job has its own workflows, conventions, software systems, and exception handling, so agents must learn on the job to specialize. That gap is widening, and open models face a capability wall they need to break through.

Original post →

More from AGI Musings

AGI Musings channel →