Questioning AI Safety's Capability Extrapolation
AdrienLE · x · 2026-07-12
The author shared a commentary, noting a recurring pattern in "doomer" AI safety research:
- First, treating a human "talent" (like persuasion, discernment, or charisma) as a single scalar;
- Assuming this ability is primarily determined by intrinsic individual factors, rather than context or environment;
- Hypothesizing that this scalar can be infinitely scaled up;
- Using this to extrapolate the future of AI.
They give the example that concepts like "superhuman persuasiveness" might not hold up, because "persuasiveness" may not be a stable human talent, but rather an attribute retroactively credited to an individual during a social process.
More from AGI Musings
- Humanoid robot sorting packages in a warehouse sparks debate over job loss — MonaJalal_ · 2026-07-22
- You can outsource thinking, but not understanding, in the age of agents — Yuchenj_UW · 2026-07-22
- India’s multilingual LLM edge, once obvious, is gone, the post argues — kmeanskaran · 2026-07-22
- AI media may be cleaned up with provenance tracking, notes, and prediction markets — NathanpmYoung · 2026-07-22
- Ryan Greenblatt says economists underestimate AI’s growth impact even in a 100 million worker scenario — RyanGreenblatt · 2026-07-22
- Peter Diamandis says experts in the old world are often last to see the new one — PeterDiamandis · 2026-07-22