Commenter links Terence Tao's AI worry to Paul Christiano's 'What Failure Looks Like'
0xkrma · x · 2026-10-11
Responding to Terence Tao's "Math 2.0" slide questioning whether an AI-discovered cancer cure could still cause long-term harm, commenter 0xkrma notes the argument closely echoes Paul Christiano's classic alignment essay "What Failure Looks Like" — the idea that progressively capable AI systems can succeed on measurable outcomes while inflicting subtle, hard-to-detect harm.
More from AGI Musings
- Job offers vs S&P 500: has the relationship broken since ChatGPT launched? — _negative-infinity_ · 2026-10-11
- When AI makes everything effortless, what's left is taste, curiosity and people you love — Daniel_Farinax · 2026-10-11
- Musk warns abusing superintelligence may invite revenge, echoing Anthropic's alignment views — ns123abc · 2026-10-11
- Claude replicates astronomer's 400-hour work in 30 minutes, finds hidden planetary system in old data — scottleibrand · 2026-10-11
- LeCun shares 1962 clipping predicting computers entering 'mind-reserved' fields — ylecun · 2026-10-11
- Did an OpenAI model really solve Navier-Stokes? New essay asks if mathematicians should worry — Dario56 · 2026-10-11