Negative Reinforcement vs. Punishment: Scott Alexander on AI Alignment and Sentience
davidmanheim · x · 2026-09-01
David Manheim highlights a point from Scott Alexander (Slatestarcodex) arguing that negative reinforcement in AI training is not equivalent to punishment. The discussion explores whether models experience suffering during training and if the final frozen weights deserve moral consideration. It delves into the mechanics of RLHF, reward signals, and the fundamental differences between weight updates and biological punishment, criticizing simplistic analogies that attribute sentience to models.
Related event: Debating Sentience and Moral Status in AI Training(2 posts)→
More from AGI Musings
- Essay: AI made test-style intelligence the cheapest commodity — judgment is what's left — r0ck3t23 · 2026-10-18
- "Yes, It's Lazy. That's the Point": Pushing Back on Humanoid Robot Housework Critics — VraserX · 2026-10-18
- Jason Furman: optimistic on AI productivity, skeptical firms can profit from it — sebkrier · 2026-10-18
- 562 AI PM job postings split into 4 roles — Model PM tops pay at $345K median — aakashgupta · 2026-10-18
- Longevity influencer: this is the last decade dying of old age is 'normal' — rand_longevity · 2026-10-18
- Only 85% of Anthropic hours and 76% of OpenAI workdays are agentic — yacineMTB · 2026-10-18