Decision theory debate: predictably one-boxing an Omega stupider than you
jessi_cata · x · 2026-10-28
On 'logical counterfactuals', the author argues the most believable decision-theoretic motive is the 'agent simulates predictor' scenario: how can you predictably one-box an Omega who is stupider than you and whom you can simulate? You might benefit from reasoning about stupider mind-states full of logical uncertainty, and predictably-to-them taking the action you want them to predict you take. But once the agent simulates the predictor, it resembles transparent Newcomb, raising open questions: overall one-boxing probability or conditional on seeing an empty/full box, and how fixed points are resolved. In the thread, the author says they know of no specific problem requiring logical counterfactuals and doesn't believe they exist — though neither claim is easy to prove.
Related event: Decision Theory Researchers Question Logical Counterfactuals(2 posts)→
More from AGI Musings
- Frontier Models as CapEx, Open Models as OpEx: Where AI Token Spend Is Heading — curious_vii · 2026-10-28
- Chris Albon's thought experiment: how would AI history differ if another community found scaling? — chrisalbon · 2026-10-28
- AAAI/ICLR Reviewer: About Half of Reviews Look AI-Generated, Peer Review in Crisis — dsmirza · 2026-10-28
- Toby Walsh's 'God AI': Past Tech Automated Muscles, AI Automates Decisions — TobyWalsh · 2026-10-28
- Reddit debate: AI curing baldness vs humanoid robots giving autonomous haircuts — which comes first? — Neurogence · 2026-10-28
- Risk researcher: policy briefings still recapping what Yudkowsky argued a decade ago — davidmanheim · 2026-10-28