AI veteran: no autonomous system has ever adapted to novelty — humans were always the fallback
tdietterich · x · 2026-09-21
In a thread on autonomous agent safety, researcher Thomas Dietterich cites ddwoods2's career-long work on "automation surprises": we have never built an autonomous system that adapts to novelty — human operators always provide the adaptive capability. Agent designers hope this time is different, that agents can reason about novel failures and environmental changes, but testing such systems is extremely hard. Component failure modes and environmental changes exhibit unbounded variety; only narrow tasks in unchanging environments are exceptions.
Related event: Dietterich: Autonomous Systems Still Need Human Oversight(2 posts)→
More from AGI Musings
- Researcher argues for 'public AI': making AI trajectories searchable and minable beyond big labs — const_reborn · 2026-09-21
- Weighing lives: does AI-accelerated medical research outweigh existential risk? — MaxUnfried · 2026-09-21
- AI computer use will break social networks' walled gardens, argues X user — tokumin · 2026-09-21
- Giles Carden and Josh Freeman discuss AI and the future of universities — ArtificialOther · 2026-09-21
- Back-of-envelope math: 60k ICLR submissions equal roughly one per AI PhD student worldwide — kchonyc · 2026-09-21
- Nature Health Paper Proposes an 'Epidemiology of AI,' Arguing AI Is Now a Determinant of Health — EricTopol · 2026-09-21