Juan Benet asks if valuing diversity of consciousness follows from knowledge growth
juanbenet · x · 2026-09-30
Protocol Labs founder Juan Benet engaged with an AI alignment article's principles, questioning whether principle 3 (valuing diversity of consciousnesses) can be derived from principle 1 (growth of knowledge) — a proof would tie into the orthogonality thesis and reduce many risks. He frames it as finding an attractor basin in capabilities/goals space where capability increases stay within a goal subspace, noting it requires agent architectures where knowledge can affect values, as in humans and current LLMs.
More from AGI Musings
- Agentic CPU Guidebook: agent workloads to drive server CPU market to $221B by 2030 — BenBajarin · 2026-09-30
- Ex-OpenAI researcher Mina Lee launches Substack on writing with AI without losing your soul — MinaLee__ · 2026-09-30
- Waymo Fails 1 in 20-30 SF Rides, Signaling Physical Job Automation Is Further Off — herbiebradley · 2026-09-30
- Daniel Lemire: AI Is Eroding Software Moats, Forcing Vendors to Compete for Users — lemire · 2026-09-30
- RentAHuman launches: AI agents hiring real humans for street marketing — mathemagic1an · 2026-09-30
- Peter Diamandis talks art vs science with Neal Stephenson and Mira Lane — PeterDiamandis · 2026-09-30