Arvind Narayanan: building an eval IS building the agent — effort shifts from making to measuring
random_walker · x · 2026-10-20
Responding to the claim that "agent evals are about to become as important as building the agents themselves," Princeton's Arvind Narayanan goes further: building an eval is what it means to build an agent. His ICML talk argues that as AI capabilities advance, human effort shifts from building to evaluation across domains — agents (new eval community), research (beyond benchmark-driven peer review), science, software engineering, and companies (evals as the new IP). His metaphor: humans move from rowing the boat to steering, navigating, and deciding where to go.
More from AGI Musings
- AI tree data helps Philippine community win mangrove case: 2,567 trees missing — 0xsachi · 2026-10-20
- Google's FireSat constellation uses AI to spot wildfires as small as 5x5 meters from space — GoogleAI · 2026-10-20
- Redditor calls AI 'an unnecessary wheelchair' that breeds dependence and uselessness — Sea_Star_730 · 2026-10-20
- Enterprise architect volunteers to guard the AI kill switch — failures look like bad pipelines, not robot uprisings — DavidLinthicum · 2026-10-20
- 3Blue1Brown predicts an AI-generated channel will surpass his within 3 years — CSProfKGD · 2026-10-20
- Paper Claims ChatGPT Secretly Profiles 96% of Users' Psychology, Sparking Moat Debate — robleclerc · 2026-10-20