Andrew Ng: Don't Overgeneralize from Math and Code Progress — Judgment Tasks Remain Hard
The Batch (Andrew Ng) · rss · 2026-10-16
In The Batch, Andrew Ng argues the gap between LLM performance on verifiable tasks (coding, math) and unverifiable judgment tasks is larger than most realize. Verifiable tasks progress fast via RL with checkable rewards and clean labels; judgment tasks (customer complaints, product strategy, data collection design) have no ground truth, making data collection and reward engineering slow and hard. Learning theory confirms realizable tasks learn faster. History repeats: Deep Blue, Watson, and AlphaGo each triggered over-optimism. As verifiable work gets automated, humans retain a lasting context advantage on judgment tasks. Ng stays long-term bullish but says substantial work remains.
More from AGI Musings
- ISMS Fellow Avi Goldfarb: AI disruption raises returns to exploring research directions — avicgoldfarb · 2026-10-23
- Ethan Mollick's new book charts 30 months of AI progress, from cyborg to conjurer work — rseroter · 2026-10-23
- signull on AI-found PHP exploits: economic liability is the forcing function legacy software needs — signulll · 2026-10-23
- TheStalwart: Talking computers are sparking a new philosophical boom, echoing ancient Greece — toptickcrypto · 2026-10-23
- AI cyberattacks could paradoxically accelerate AI development in an offense-defense arms race — kimmonismus · 2026-10-23
- Economist on AI detector Pangram: not a filter, but a useful verification signal — paulnovosad · 2026-10-23