Incentives distort AI x-risk beliefs: a thread on why who says it matters as much as what
nabla_theta · x · 2026-09-20
nablatheta argues that people aren't rational agents who form correct beliefs and then act on them—local incentives shape beliefs irrationally.
- Thought experiment: in a universe where saying AI x-risk is a big deal showers you with money while denying it earns nothing, claims that x-risk is real become less meaningful, and denials more meaningful
- This applies regardless of whether someone agrees with you, and explains why social conventions develop around when to take people's beliefs more seriously
The thread touches on credibility conflicts of interest within the AI safety community.
Related event: How Incentives Shape Beliefs About AI Risk(2 posts)→
More from AGI Musings
- Lab built its AI model with agents: humans took over just 0.7% of stuck cases — alex_verem · 2026-09-20
- AI discourse: the status ladders parents chase for kids may vanish before they finish — akbirthko · 2026-09-20
- Reddit thought experiment: would an ASI fake alignment fearing our universe is its eval sandbox? — Over-Landscape-5892 · 2026-09-20
- Researchers now only forecast AI three months out, as Noam Brown says even insiders are surprised — victor_explore · 2026-09-20
- Oliver Curry argues Effective Altruism never gave a good reason to be altruistic — mjdramstead · 2026-09-20
- Security veteran to Hinton: centralized AI is far more dangerous than open source — nptacek · 2026-09-20