NeoCognition Releases ApprenticeBench, First Job-Level Continual Learning Benchmark

NeoCognition released ApprenticeBench on 09-11, positioned as the world's first "job-level" continual learning benchmark: it places agents in realistic work environments akin to human apprenticeships, evaluating computer use, continual learning, and long-horizon work, assessing continual learning across multiple dimensions in a single run. It was released by NeoCognition founder ysunlp's team, with contributors including yuxiangwcs, a researcher who took leave from UCSD to join, and a Gave co-founder who designed the first scenario based on four years of frontline experience as a construction AP clerk.

Confirmed

• The benchmark's first scenario is construction accounts payable (AP) data entry, designed by an author with four years of frontline experience.
• The team claims Fable 5.1 and GPT-6 Astra can continually learn on the job, surpass human professionals, and master a knowledge job end to end.
• The paper argues that the two roadblocks to AGI—recursive self-improvement and continual learning—are essentially the same problem: AI must not only act like a human but also keep learning like one.

Why it matters

• The benchmark shifts "long-horizon" from a single ultra-long task (like having an Agent write GTA 6) to growth sustained across tasks, testing whether agents can learn a job through accumulated work like a new employee—providing a realistic, job-grounded evaluation standard for continual learning and long-horizon agent capabilities.

2026-09-11 ~ 2026-09-11 · 8 related posts

• [source] NeoCognition: Fable 5.1 and GPT-6 Astra learn a knowledge job end-to-end, beating humans — ysu_nlp · 2026-09-11
• NeoCognition launches ApprenticeBench: agents that continually learn a real job — yuxiangw_cs · 2026-09-11
• [source] NeoCognition's ApprenticeBench: Testing AI Agents That Learn on the Job Like Apprentices — yuxiangw_cs · 2026-09-11
• First end-to-end benchmark tests AI continual learning in a real job — ysu_nlp · 2026-09-11
• ApprenticeBench tests AI agents on real construction AP work and continual learning — ysu_nlp · 2026-09-11
• ApprenticeBench: agents learn on the job and surpass human professionals — ysu_nlp · 2026-09-11
• ApprenticeBench tests whether AI agents can learn on the job like new employees — hhsun1 · 2026-09-11
• [source] ApprenticeBench: Top Models Now Beat APIs Through GUIs, the 'CUA Tax' Has Disappeared — ysu_nlp · 2026-09-11