Erik Hoel: provably non-conscious AI would permanently reserve parts of the economy for humans

erikphoel · x · 2026-11-19

Responding to the debate that human primacy is hard to reconcile with AIs surpassing humans in every respect, Erik Hoel offers a definable scenario: if AI were engineered to be non-conscious, only humans would be moral patients, resolving the tension. Per the quoted essay, non-conscious AIs cannot be ends in themselves, are nonsensical subjects for natural personhood rights, and remove impetus to replace humanity with a successor species. Bonus: the "relational economy" — jobs where the producer's identity and consciousness are instrumental, like client relationships, authors, creators, community leaders — would be permanently reserved for humans, since non-conscious AI can't provide genuine conscious connection.

Original post →

More from AGI Musings

AGI Musings channel →