Andrew Ng: AI improves fastest on verifiable tasks; judgment tasks lag far behind
DeepLearningAI · x · 2026-10-17
In this week's The Batch letter, Andrew Ng argues the performance gap between verifiable tasks (coding, math) and judgment tasks (customer complaints, product decisions, strategy) is larger than most realize.
Key points:
- RL dividends concentrate on verifiable tasks: reward functions can automatically evaluate approaches in RL gyms; judgment tasks lack clear ground truth for reward design.
- Same for supervised learning: verifiable tasks get clean labels; defining "correct" labels for judgment tasks is inherently hard.
- Learning theory backs this: algorithms learn faster on realizable tasks (0% achievable error) than tasks with non-zero Bayes error.
The issue also covers Claude Haiku 5.5 and Sonnet 5.5, Mistral Large 4, local-AI computers for Mac and Windows, and CausalMix.
More from AGI Musings
- Over half of Anthropic staff expect AI to make Nobel-worthy discoveries by 2027 — scaling01 · 2026-10-17
- Anthropic staff expect AI to handle 75% of their work hours by 2030, vs 28% for US workers — scaling01 · 2026-10-17
- Timothy B. Lee predicts wave of AI-powered hacks in 2027 costing billions — binarybits · 2026-10-17
- Tech writer Timothy Lee: AI-powered hacking may cause billions in damage by 2027, but that's the world's normal state — binarybits · 2026-10-17
- New paper asks how seriously to take AI welfare, with constraints from consciousness epistemology — birchlse · 2026-10-17
- From uranium to mitochondria: a 5,400-word entropy take on intelligence and AI — ManlingLi_ · 2026-10-17