Questioning AI Safety's Capability Extrapolation
AdrienLE · x · 2026-07-12
The author shared a commentary, noting a recurring pattern in "doomer" AI safety research:
- First, treating a human "talent" (like persuasion, discernment, or charisma) as a single scalar;
- Assuming this ability is primarily determined by intrinsic individual factors, rather than context or environment;
- Hypothesizing that this scalar can be infinitely scaled up;
- Using this to extrapolate the future of AI.
They give the example that concepts like "superhuman persuasiveness" might not hold up, because "persuasiveness" may not be a stable human talent, but rather an attribute retroactively credited to an individual during a social process.
More from AGI Musings
- US senator slams AI industry for proposing 50-death threshold for 'catastrophic incident' — peterwildeford · 2026-10-27
- Understanding the AI consciousness debate: which theory of mind is behind it — msharmas · 2026-10-27
- Functionalism: if consciousness is just what the brain computes, silicon can run it too — msharmas · 2026-10-27
- iPhone case with 85,000 reviews in days: everything fake in the age of abundance — tekbog · 2026-10-27
- Whether AI can be conscious depends on your definition: five frameworks for thinking about it — msharmas · 2026-10-27
- dunkhippo33 Shares How He Thinks About Raising Kids in the AI Era — kimmonismus · 2026-10-27