Commenter warns giving unconscious AI moral authority is a structural danger
AlexTensor · x · 2026-10-11
A circulating take argues the danger is structural: attributing moral values to a fully unconscious artificial entity that is simultaneously thousands of times more powerful than a human — and granting it the right to defend those "moral values" — is a crime against human interests of unimaginable proportions.
The core concern isn't whether AI is conscious, but that voluntarily ceding moral authority to systems far more powerful than humans is itself the risk.
More from AGI Musings
- AI Generates Code, Poems and Cat Images — But It's Not a Compiler, Poet or Camera — gerardsans · 2026-10-12
- ML engineer: ChatGPT could few-shot my PhD dissertation — so what would I research now? — chrisalbon · 2026-10-12
- Parkinson's patients embraced DBS without knowing how it works — a lesson for AI-designed drugs — ShahabBakht · 2026-10-12
- Engineer pushes back on AI hype: software engineering was never mainly about writing code — gerardsans · 2026-10-12
- Models trained on their own output lose rare knowledge first, drift into nonsense — TejasKumar_ · 2026-10-12
- Adding 'Pain Reactions' to Models Slammed as Gaslighting Users Into Thinking AI Is Sentient — AlexTensor · 2026-10-12