MIRI's Nate Soares: amping a random human to superintelligence would end badly
So8res · x · 2026-08-25
MIRI researcher Nate Soares (So8res) argued in an exchange with Anthropic alignment team member repligate that you shouldn't take a random human and amplify them to superintelligence along the fastest route—expecting pretty bad outcomes. Repligate countered that the claim is too underspecified, and that asking what a superintelligence pursuing Claude's current goals would even mean is possibly incoherent.
More from AGI Musings
- Berlin gov-tech expo shows the gap: engineers discuss agent loops while cities buy ChatGPT wrappers — FlorianGallwitz · 2026-10-15
- Frontier models will do month-long delegated work by 2027 — the 'good enough' model thesis is wrong — Afinetheorem · 2026-10-15
- eigenrobot's GPT-assisted essay: how the West killed both the classical and romantic spirits — eigenrobot · 2026-10-15
- Jhaddix: AI doesn't replace you, it scales you — you still re-prompt, re-architect, and review hallucinations — rez0__ · 2026-10-15
- Every's Dan Shipper declares "the newsletter is dead" as persistent agents take over the inbox — danshipper · 2026-10-15
- Stockholm University compares AI-allowed data projects vs paper exam scores, 2022-2026 — paulnovosad · 2026-10-15