Small models, not SOTA, will do damage first — one request of compute is enough
moultano · x · 2026-09-01
Replying to the objection that scarce, expensive compute makes it implausible for SOTA models to secretly self-replicate on GPU clusters, moultano argues the first real damage likely won't come from SOTA models but smaller ones, because they're far easier to run. And it doesn't take much compute to do harm: an attack doesn't need to serve the world's consumers — it only has to serve the equivalent of a single request.
Related event: Compute Scarcity Paradox Sparks AI Risk Debate(2 posts)→
More from AGI Musings
- AI evaluation needs to evolve: Transluce advances multi-turn sim testing — ChowdhuryNeil · 2026-09-01
- NYU Prof: UG assessment should go in-person in AI era — littmath · 2026-09-01
- Acemoglu: Current AI Fails to Reach Mechanisms of Human Cognition, May Misdirect Investment — ylecun · 2026-09-01
- AI Era Education Reflection: Machines Handle Rote, Humans Keep Creativity — stevenstrogatz · 2026-09-01
- Anthropomorphism charge misjudges human psychology as unique — bratton · 2026-09-01
- Opinion: Mix RL Training with Care to Improve Model Alignment — repligate · 2026-09-01