FDT decision-theory algorithm shown to defect in almost-twin prisoner's dilemma
jessi_cata · x · 2026-11-07
jessicata demonstrated that FDT — the graph-based decision theory algorithm from the FDT paper, not the "vibes-based" version — defects in the almost-twin prisoner's dilemma, sharing a ChatGPT conversation as the demo. The quoted thread explains why: FDT builds a quasi-causal graph where "your decision node" depends on "your observation," so different voters seeing different observations have decision nodes that don't quasi-cause each other. FDT therefore reasons as if intervening on its own decision given its observation, and that intervention won't quasi-causally affect other voters — leading it to defect. Technically FDT is underspecified since it doesn't say how to build the graph, and building a sensible graph requires hard "computational logical counterfactuals."
More from AGI Musings
- Sandberg: experts in 1925 mistook ignorance about atomic energy for proof of difficulty — anderssandberg · 2026-11-07
- Sandberg dissects the 1940 argument that harnessing atomic energy was impossibly remote — anderssandberg · 2026-11-07
- Researcher: the mob cared about citations until LLMs got good at them, then stopped instantly — RexDouglass · 2026-11-07
- AI Safety Researcher Vows to Stay, Calls on OpenAI and Anthropic to Stop RSI — trevposts · 2026-11-07
- AI Safety Researchers List Concrete Ways to Contribute Beyond Fellowships Like MATS — AndyMasley · 2026-11-07
- Garrison Lovely: data center moratoriums won't slow AI, chip export controls would — GarrisonLovely · 2026-11-07