Debate: human behavior is instrumentally constrained, but not Bellman-equation backchaining
enormochungus · x · 2026-08-30
enormochungus and jdpressman debate instrumental vs terminal behavior. He argues all behavior is instrumental, but tactic choice is heavily constrained by environment, cognitive architecture, and intelligence; humans' reputations and long interaction histories create Newcomblike constraints that blur signaling versus real preference. He distinguishes the Yudkowskian paperclipper (no value) from anthropoid robot successors (some value) as different flavors of "instrumental convergence". jdpressman replies that we face a vast space of possible goals rather than minds, which doesn't undermine doom arguments—humans wiped out Neanderthals after all.
More from AGI Musings
- Duke Study of 4,439 People Finds Coworkers Quietly Rate AI Users as Lazier — aakashgupta · 2026-10-16
- DSPy Author's $70 O'Reilly Book Rejected by Engineer Who Now Just Tells Claude What to Do — lateinteraction · 2026-10-16
- 'A machine proving new math, a car driving up a wall': AI researcher on how weird it's getting — rand_longevity · 2026-10-16
- Yacine on training: 'Neural nets are lazy stupid idiots — I build hells to make them ubermensch' — yacineMTB · 2026-10-16
- Jeff Clune: Moving the Goalposts Will Be Humanity's Last Job — jeffclune · 2026-10-16
- Replit CEO Amjad Masad Says the Future of Work Is the 'Self-Driving Company' — TechCrunch AI · 2026-10-16