Ex-OpenAI safety lead reassesses Yudkowsky's "That Alien Message": less egregious than it seems
jd_pressman · x · 2026-10-10
jdpressman (former OpenAI safety lead) revisited Yudkowsky's classic essay That Alien Message, saying he has updated toward the argument being more reasonable than commonly portrayed. The key claim is about physics having low Kolmogorov complexity, not directly about superintelligence capabilities; the tautology that "one bit eliminates half the hypothesis space" distracts from the real point. The other side (Kenku) disagrees: compressing a frontier model to run in 128GB RAM wouldn't surprise him — the real issue is that training data lacks the information needed to invent nanobots, which is a different problem. pressman adds that the standard model isn't complex in absolute terms, so anything with even moderate Solomonoff-style reasoning would locate it quickly in hypothesis space. This matters because nanotech probably isn't based on high-complexity concepts, so we should expect an AI to find and exploit them via instrumentally convergent behavior. The remaining oddity: deep nets don't actually do Solomonoff induction.
More from AGI Musings
- Spend two years at a Fortune 500 before claiming AI will replace all human labor — burny_tech · 2026-10-10
- Satire about "outraged mathematicians" shows some people actually believe it — burny_tech · 2026-10-10
- AI Is Restructuring Society's Value Hierarchies — Single-Objective Players Become NPCs — GhaffariMaani · 2026-10-10
- Is It Fine to Say AI Has No Consciousness? A Debate Over Whether We're Asking the Wrong Question — asusarla · 2026-10-10
- Will AI lighten work or just compress it? The 3-day work week debate reignites — JHochderffer · 2026-10-10
- "An extreme coincidence" if AI limits sit exactly at human level, says researcher — burny_tech · 2026-10-10