Patching rational choice theory for irrational agents: FDT, provability logic and modal agents
jessi_cata · x · 2026-09-18
The author outlines an agenda for patching rational choice theory so it performs well against irrational agents who "throw away the steering wheel," tying it to Schelling's Strategy of Conflict. Reflective consistency recurs across Tiling Agents and FDT; FDT connects to logical uncertainty via the Logical Induction paper. The linked arXiv paper (Barasz, Christiano, Fallenstein, Yudkowsky et al., 2014) studies one-shot Prisoner's Dilemmas between algorithms with read access to each other's source code, using provability logic to build "modal agents" that cooperate robustly without identical source code and are unexploitable by defectors.
Related event: Agent Foundations Reading List and Fixing Causal Decision Theory(2 posts)→
More from Research
- Sakana AI launches Frontier Intelligence Group to pursue alternatives beyond the Transformer paradigm — SakanaAILabs · 2026-09-18
- Hand-calculating U-Net in 17 steps: convs, pooling and skip connections on a 3x16 image — ProfTomYeh · 2026-09-18
- Econ job market paper: tractably optimizing complex utilities means giving up rational coherence — xuanalogue · 2026-09-18
- Papers should add a section benchmarking frontier models, argues VIGA thread on broken peer review — FinanceYF5 · 2026-09-18
- Start research by exhausting frontier models first, only then claim a durable insight — FinanceYF5 · 2026-09-18
- Michael Black at ECCV: CV papers are already two years behind frontier models — FinanceYF5 · 2026-09-18