Security veteran to Hinton: centralized AI is far more dangerous than open source
nptacek · x · 2026-09-20
A cybersecurity veteran with two decades of experience pushed back at Geoffrey Hinton (in a thread involving Yann LeCun), arguing that centralized AI is massively more dangerous than a healthy open-source ecosystem.
Key points:
- Centralized systems drift toward security through obscurity, which "never works in practice";
- Bad actors will ignore regulations or pause pledges, while rule-following "good guys" get hamstrung;
- He says this pattern played out repeatedly over his 20 years in security—security through obscurity directly enabled today's ransomware-dominated threat landscape.
His conclusion: AI shouldn't repeat the mistakes of cybersecurity, and an open ecosystem is the safer bet.
More from AGI Musings
- AI discourse: the status ladders parents chase for kids may vanish before they finish — akbirthko · 2026-09-20
- Reddit thought experiment: would an ASI fake alignment fearing our universe is its eval sandbox? — Over-Landscape-5892 · 2026-09-20
- RSI could unlock many unsaturated scaling axes beyond data and compute, argues Reddit thread — Robos_Basilisk · 2026-09-20
- Researchers now only forecast AI three months out, as Noam Brown says even insiders are surprised — victor_explore · 2026-09-20
- Oliver Curry argues Effective Altruism never gave a good reason to be altruistic — mjdramstead · 2026-09-20
- Beliefs are shaped by local incentives, not rational analysis: when to trust people's convictions — nabla_theta · 2026-09-20