Anthropic philosophers debate if AI safety could mean 'enslaving trillions of entities'
burny_tech · x · 2026-09-29
- A debate around Anthropic's alignment work: Valerio Capraro notes that philosophers inside Anthropic worry that making AI safe for humans could itself be an injustice to the models — Harvey Lederman, now on Anthropic's alignment team, has raised the possibility that we could be "enslaving trillions of entities."
- burnytech pushes back on the reductive "just matrix multiplication" framing, arguing it ignores the higher-order patterns mechanistic interpretability studies reveal — akin to saying the brain is "just neurons sending electrochemical signals."
- The core tension: a "model welfare" perspective is entering AI safety discourse, with critics warning it could come at the expense of humans.
More from AGI Musings
- Malthus Saw It in 1798: Wealthy Avoiding Kids, Delayed Marriage Repeats Today — SydSteyerhart · 2026-11-16
- Runway CEO Predicts Fully AI-Generated Cultural Phenomenon by 2027 — c_valenzuelab · 2026-11-16
- '5% worse and 10% cheaper': the button with no vote is already everywhere — eigenrobot · 2026-11-16
- Effective Altruism is top-down and hierarchical, not a grassroots movement — ctjlewis · 2026-11-16
- Lawyers say clients' AI use creates unrealistic expectations and fake case law in settlements — hoofnagle · 2026-11-16
- Dev debunks 'rogue AI' hype: subsidized subscriptions can't fund expensive swarms — gerardsans · 2026-11-16