"Morality is crystallized intelligence": norms come from selection, not raw intelligence
analisereal · x · 2026-10-19
In the debate between Jack Clark and Aaron Hall on whether superintelligence's moral reasoning is legible to humans, analisereal offers a third angle: most morality is arguably crystallized intelligence shaped by environment — norms that stuck around because the groups holding them thrived. On that premise, morality is shaped mainly by the selection process, not raw intelligence.
This speaks to Hall's doubt that higher intelligence implies greater moral wisdom, and reinforces Clark's legibility point: a smarter AI's normative judgments are not automatically more moral.
Related event: Debate: Will Superintelligence's Moral Reasoning Be Human-Readable?(2 posts)→
More from AGI Musings
- SciΠvot benchmark: frontier models prefer flawed research plans and lag human scientific judgment — Youran Sun · 2026-10-19
- Veteran game dev: AI frees creators from publishers where approval odds were under 5% — majidmanzarpour · 2026-10-19
- Indie dev: AI has flooded the market with 100x more plugins, forcing business model rethink — DeliciousOrangina · 2026-10-19
- Dario Amodei Sees AI Beating Humans at Everything in 2-3 Years; Marcus Scoffs — GaryMarcus · 2026-10-19
- Grady Booch corrects Ethan Mollick: SAGE interceptor assignment was human, not automated — Grady_Booch · 2026-10-19
- Sabine Hossenfelder: Stop requiring human understanding—AI is the way forward for science — skdh · 2026-10-19