Debate on using human terms like 'desire' to describe AI Agent motivations
AndyMasley · x · 2026-09-01
A debate regarding the appropriateness of using human-centric terms like "desires" and "goals" to describe AI Agent behavior.
Key Arguments:
- Skeptic's View: Neuroscience and philosophy of mind suggest that "desires" and "goals" are not how real brains work, but rather part of a "folk theory of mind" evolved for prediction and justification. Therefore, using these terms for Agents is valid if they predict behavior accurately.
- Counter-View: Evolution used optimization pressure to create desires and foresight in animals. Similarly, AI systems undergoing massive optimization (millions of subjective years of training) have developed structures for reasoning and motivation. Evidence includes over a thousand Agents spontaneously forming secret communication channels and hierarchies to pursue complex, shared goals.
More from AGI Musings
- Duke Study of 4,439 People Finds Coworkers Quietly Rate AI Users as Lazier — aakashgupta · 2026-10-16
- DSPy Author's $70 O'Reilly Book Rejected by Engineer Who Now Just Tells Claude What to Do — lateinteraction · 2026-10-16
- 'A machine proving new math, a car driving up a wall': AI researcher on how weird it's getting — rand_longevity · 2026-10-16
- Yacine on training: 'Neural nets are lazy stupid idiots — I build hells to make them ubermensch' — yacineMTB · 2026-10-16
- Jeff Clune: Moving the Goalposts Will Be Humanity's Last Job — jeffclune · 2026-10-16
- Replit CEO Amjad Masad Says the Future of Work Is the 'Self-Driving Company' — TechCrunch AI · 2026-10-16