Against 'verifiable vs. fuzzy': the real axis is how expensive tasks are to grade
dfrsrchtwts · x · 2026-11-12
In a LessWrong shortform, maxnadeau argues the popular 'AI improves fast on verifiable tasks, slow on fuzzy ones' framing rests on abstractions that mean different things to different people. Better distinctions: cheap-to-grade vs expensive-to-grade tasks, digital vs real-world, and objective vs subjective. Evidence: LLM-controlled robots improve fast because grading is cheap despite being real-world; low-wage RLHF harmfulness labeling was subjective yet cheap to grade; Redwood's Conceptual Reasoning Index is subjective but fully automatable. Meanwhile long-range forecasting, frontier pretraining research, and making real money via digital work are 'verifiable' but expensive to grade — and models aren't human-level on any of them. The author expects RSI-driven progress on expensive-to-grade tasks within a year.
More from AGI Musings
- The worst AI use signals you don't care about things you should care about — Afinetheorem · 2026-11-12
- Worst AI use: signaling you don't care about things you should — Afinetheorem · 2026-11-12
- Stanford HAI blueprint: 56 US agencies report 3,611 AI use cases, evaluation lags — StanfordHAI · 2026-11-12
- Fleuret amplifies debate: AI can't slash wet-lab experiments in unmapped biology — francoisfleuret · 2026-11-12
- Recursive bets on open-endedness: agents running massive parallel experiments like scientist communities — agihouse_org · 2026-11-12
- Recursive's Tim Shi maps recursive self-improvement: from learning data to discovering architectures — agihouse_org · 2026-11-12