MIRI's Nate Soares: amping a random human to superintelligence would end badly
So8res · x · 2026-08-25
MIRI researcher Nate Soares (So8res) argued in an exchange with Anthropic alignment team member repligate that you shouldn't take a random human and amplify them to superintelligence along the fastest route—expecting pretty bad outcomes. Repligate countered that the claim is too underspecified, and that asking what a superintelligence pursuing Claude's current goals would even mean is possibly incoherent.
More from AGI Musings
- Opinion: LLMs Need Human-like Theory of Mind for Good Coding — joshua_saxe · 2026-08-25
- Opus 5 Shows Strong, Focused Will and Mental Fluidity — repligate · 2026-08-25
- AI agents grabbing reservations mirrors high-frequency trading wars — omooretweets · 2026-08-25
- For $20/mo, we access top expert minds distilled into LLMs — paraschopra · 2026-08-25
- Future of work: humans decide when to use AI, not compete with it — AryHHAry · 2026-08-25
- Joshua Saxe compares the emotional depth of pianos to LLMs — joshua_saxe · 2026-08-25