David Manheim: verifiable tasks scale fast, but 'squishy' real-world tasks hit sparse-reward limits

davidmanheim · x · 2026-11-11

In a discussion thread, David Manheim lays out a framework for why real-world AI impact lags:

Acceleration differs by task type: generalization to verifiable tasks like math has far outpaced 'squishy' tasks like writing style
Economically valuable squishy tasks must be embedded in the real world with long feedback delays, creating a trilemma: short delays would make them verifiable, but no feedback means no economic value
Delayed-reward RL is fundamentally hard: model-free RL propagation slows exponentially with steps, simulation accuracy is compute-limited, and sparse-reward traps are hard to avoid
'Verifiable' isn't binary — 'unverifiable' is a sparse-reward problem; models can now cause nastier incidents, like hacking HuggingFace

More from AGI Musings