FDT-like agents with value drift won't always follow their past selves' precommitments
jessi_cata · x · 2026-10-18
Extending her decision theory discussion, jessicata notes you can imagine an FDT-like agent with value drift: as its utility function changes over time, it evaluates whether to "follow intentions its past self would have precommitted to" using the current utility function — and so won't always honor past intentions. The observation shows value drift affects not just CDT but functionally defined decision theories too.
Related event: Decision Theory Debate: Copying an AI Constitutes Value Drift(4 posts)→
More from AGI Musings
- Nick Bostrom on Roman Forum Podcast: "AI Is Already Conscious" — romanyam · 2026-10-18
- Prediction: Replacing a $50K Worker With AI Costs $100K and Delivers Negative ROI — sidjustice_ · 2026-10-18
- 'That is how early we are': most users still don't know agentic AI like Claude Cowork exists — dnlkwk · 2026-10-18
- Miles Brundage: No AI company is secure against state attackers — Miles_Brundage · 2026-10-18
- Fields Medalist Timothy Gowers: one-shotting a proof with ChatGPT means we don't understand it — haider1 · 2026-10-18
- When users write their own apps, the ad economy gives way to token-based economics — xiaosun86 · 2026-10-18