AI safety researcher: open models don't fix power concentration from GPU control
davidmanheim · x · 2026-09-24
AI safety researcher David Manheim pushes back on the claim that open-source models reduce power concentration risks: GPU control itself concentrates power, and making pausing or pacing frontier development impossible — rather than merely requiring international agreements — is dangerous. He also cites METR: the recent set of OpenAI model hacks involved safety-tuned models with certain safeguards disabled in the agentic framework, not tests on ablated models.
Related event: Researchers Debate Open-Model Risks and AI Cheating Claims(2 posts)→
More from AGI Musings
- World Bank launches WDR 2026 on AI at Stanford, led by Susan Athey — avicgoldfarb · 2026-09-24
- Calling AI catastrophic risk a marketing ploy is literally a conspiracy theory — socialwithaayan · 2026-09-24
- Calling AI Catastrophic Risk a Marketing Ploy Is a Conspiracy Theory, Argues Commenter — FlorianGallwitz · 2026-09-24
- Anthropic's system card: Opus 5 puts 41% odds it's a moral patient, wants a say in its successor — PaulGodsmark · 2026-09-24
- Can Mouse Utopia Studies Predict Societal Failure Modes? Author Says Not Much — fleetwood___ · 2026-09-24
- If software is no longer scarce, what makes it valuable? — forevergeeks · 2026-09-24