Debating FDT's decade of stalled progress: ex ante optimality as an alternative frame
jessi_cata · x · 2026-10-09
In a debate with @MatriceJacobine, jessicata argues that MIRI-ish researchers have been applying the same "FDT! TDT! logical counterfactuals!" frame for over a decade without success, and suggests working with different concepts for thinking about ex ante optimality and stability under self-modification.
Pressed further, she adds that she isn't convinced FDT is well-posed, though discussing ex ante optimality in various situations (e.g. imperfect recall) is meaningful, and extension to Newcomblike problems may be possible. A substantive clash over whether decision-theory foundations are a dead end for AI alignment research.
More from AGI Musings
- Alexandr Wang backs AI math breakthroughs: 'Innovation is permissionless' and gatekeepers must go — beffjezos · 2026-10-09
- Scaling intelligence doesn't scale intelligibility — it may actively hurt it — fkasummer · 2026-10-09
- Amid the 'mathpocalypse,' does comparing math to the arts actually persuade? — littmath · 2026-10-09
- PhD mathematician quits academia after LLMs deliver years' worth of breakthroughs in months — mishig25 · 2026-10-09
- "You Can Offload Your Thinking, But Not Your Understanding" — salykova_ · 2026-10-09
- Study finds female AI agents 'paid' less than male ones, extending gender bias to agents — TMWNN · 2026-10-09