'One thought too many': where should formula-based AI decisions stop?
rasta321 · reddit · 2026-10-03
Using iRobot's scene of a robot saving the 50%-survival man over the 11% child, the author invokes Bernard Williams' 'one thought too many' to question the spread of System One decision models (routing, guardrails, reranking, research filters): individually advisory, collectively creeping toward authority. The concern isn't wrong math but calculating things—like a child's potential—that shouldn't be calculated at all. He shares his own boundary encoding (domain, authority, mechanical enforcement point) since LLM06 says avoid excessive autonomy but not how.
More from AGI Musings
- Turing laureate David Patterson: human-to-SI transition point by year's end — davidpattersonx · 2026-10-03
- Rural Chinese elderly earn hundreds monthly watching short videos — a form of UBI? — oran_ge · 2026-10-03
- Schmidhuber cites three decades of papers on formal theories of creativity and curiosity — SchmidhuberAI · 2026-10-03
- New essay series argues superintelligence will emerge, not be built — mayanicks0x · 2026-10-03
- "Continue as usual" isn't neutral: Reddit essay argues for precaution on AI consciousness — CarefulHamster7184 · 2026-10-03
- Pedro Domingos: AI is not a new species, it's a new ecosystem — pmddomingos · 2026-10-03