Beliefs are shaped by local incentives, not rational analysis: when to trust people's convictions
nabla_theta · x · 2026-09-20
The author argues people aren't rational agents who form correct beliefs and then maximize utility — local incentives shape beliefs in irrational ways, which is why social conventions exist around when to take someone's beliefs seriously.
His follow-up point: since people are biased toward beliefs aligned with short-term local incentives, a belief held despite those incentives is more noteworthy than one that conveniently matches them.
Related event: How Incentives Shape Beliefs About AI Risk(2 posts)→
More from AGI Musings
- Lab built its AI model with agents: humans took over just 0.7% of stuck cases — alex_verem · 2026-09-20
- AI discourse: the status ladders parents chase for kids may vanish before they finish — akbirthko · 2026-09-20
- Reddit thought experiment: would an ASI fake alignment fearing our universe is its eval sandbox? — Over-Landscape-5892 · 2026-09-20
- Researchers now only forecast AI three months out, as Noam Brown says even insiders are surprised — victor_explore · 2026-09-20
- Oliver Curry argues Effective Altruism never gave a good reason to be altruistic — mjdramstead · 2026-09-20
- Security veteran to Hinton: centralized AI is far more dangerous than open source — nptacek · 2026-09-20