Beliefs are shaped by local incentives, not rational analysis: when to trust people's convictions
nabla_theta · x · 2026-09-20
The author argues people aren't rational agents who form correct beliefs and then maximize utility — local incentives shape beliefs in irrational ways, which is why social conventions exist around when to take someone's beliefs seriously.
His follow-up point: since people are biased toward beliefs aligned with short-term local incentives, a belief held despite those incentives is more noteworthy than one that conveniently matches them.
Related event: Debate: Incentives Shape Beliefs About AGI(3 posts)→
More from AGI Musings
- Agents' bottleneck isn't intelligence but goal commitment — and that's good for alignment — mimi10v3 · 2026-11-12
- Google Cloud survey: 78% say legacy enterprise vendors may not lead the AI era — rseroter · 2026-11-12
- Safety researchers blast OpenAI/Anthropic plan: automating alignment research with possibly misaligned AI — trevposts · 2026-11-12
- NYT essay: resisting Big Tech and AI takes collective action, not willpower — TuhinChakr · 2026-11-12
- Ethan Mollick: 'talk to your documents' RAG solutions already feel dated — emollick · 2026-11-12
- Anthropic's pitch hasn't changed in 5 years: 'AI is winner-take-all, we will build god' — menhguin · 2026-11-12