Debate revisits Yudkowsky's That Alien Message: physics' low Kolmogorov complexity means AI could locate dangerous tech fast
jd_pressman · x · 2026-10-10
A thread between jdpressman and KenkuAllaryi revisits Yudkowsky's classic post That Alien Message. jdpressman argues the standard model is not that complex in absolute program-size terms, so any system with moderate Solomonoff-style reasoning should locate it quickly in hypothesis space — and since nanotech likely rests on low-complexity concepts too, an AI could rapidly leverage it in instrumentally convergent hostile ways. He reads the post's real claim as being about physics' low k-complexity rather than superintelligence capabilities, calling the take reasonable though deep nets don't actually do Solomonoff induction. KenkuAllaryi counters that compute cost isn't the issue (compressing a model to run on 128GB RAM wouldn't surprise him); the real gap is that training data lack the information needed to invent nanobots, which is distinct from model capability.
More from AGI Musings
- Frontier AI builders must take philosophy seriously, argues technologist — fkasummer · 2026-10-10
- Jack Clark on torn between AI's ethereal future and real life — jackclarkSF · 2026-10-10
- dbskill turns 10.5k stars into seven-figure revenue: selling skills is e-commerce — oran_ge · 2026-10-10
- Debate: corporations got personhood — why not AI? It may just work outside our structures — adamamcbride · 2026-10-10
- Only a few Chinese AI labs like GLM are pushing toward RSI, says observer — kimmonismus · 2026-10-10
- Rumors: Google, Anthropic, OpenAI already using RSI; author asks about China and compute — kimmonismus · 2026-10-10