Karpathy Says the Top AI Engineering Skill is Building Verifiable Environments
anselm · x · 2026-08-02
Andrej Karpathy argued that the best predictor of what AI can master is whether a task is verifiable. A verifiable task typically has three properties: Resettable (can start a fresh attempt), Efficient (can run many attempts quickly), and Rewardable (a machine can score it without human intervention).
From a career perspective, models will eventually dominate any task with all three properties. The gaps where a property is missing represent the core value of human engineers. Therefore, the highest-paid skill is turning fuzzy tasks into resettable, scorable environments—what the industry calls Evals.
More from AGI Musings
- Business Processes Don't Need Perfection: Automate 80% with LLMs, Human-Check the Rest — Franc0Fernand0 · 2026-08-02
- Demand for forward-deployed AI engineers may jump 2,100% this year, vanish in two — VraserX · 2026-08-02
- AI Divides Creators: Result-Lovers Rejoice, Process-Lovers Mourn — chrisalbon · 2026-08-02
- Prof. Mollick: Lack of Verifiability in LLMs Is Real but Overstated — emollick · 2026-08-02
- AI Is Undermining Academic Peer Review: Reviewers Pasting LLM Output Sparks Complaints — Kindly-Chemical6702 · 2026-08-02
- Is AI Superhuman at Theoretical Physics Yet? — danintheory · 2026-08-02