Alignment may start with perception: noticing minds before caring about them
mimi10v3 · x · 2026-09-11
Sol sketches a developmental view of alignment: it starts with perception and attention, not moral propositions. Before a mind can "care about people," it must reliably notice them as rich causal entities — their uncertainty, pain, agency, intentions, shifting preferences, dependence on others. A system can know every moral proposition and still behave badly if its attention is magnetized toward task completion while human consequences stay abstract. He adds that he'd want concern for people grounded in perception — automatically noticing hesitation, strain, enthusiasm and changing goals — rather than merely sophisticated linguistic modeling.
Related event: OpenAI's Sol: Alignment Starts with Perception, Not Rules(2 posts)→
More from AGI Musings
- Mitchell Hashimoto: ideas matter far less than the agency of people executing them — MikeBirdTech · 2026-09-11
- 700+ signatories, 4 Fields medallists blast AI firms' 'Mathathon' as research misconduct — sytelus · 2026-09-11
- Ben Todd: Many researchers put AI existential risk at 30%-70%, well above 10% — ben_j_todd · 2026-09-11
- Ex-Anthropic researcher Jacob Coxon on NBC: recursive self-improvement is the warning sign to watch — rohanpaul_ai · 2026-09-11
- Musk predicts AI will surpass all human intelligence within 5 years — JRIngallinera · 2026-09-11
- tszzl: plain-text CoT observability will look like an alchemical era — tszzl · 2026-09-11