Asking when a rational agent does the right thing is still underrated, argues AI researcher
xuanalogue · x · 2026-09-05
The author, whose work is more empirical and applied than MIRI-style agent foundations, argues it remains underrated to ask: when would a rational agent do the right thing, and under what concept of rationality?
He adds that this kind of understanding offers normative guidance now and will scale to future agents — especially in the "RL strikes back" era of AI development.
Related event: Researcher: when rational agents do the right thing remains underrated(2 posts)→
More from AGI Musings
- Anthropic builds 13M-line Lean proof of Fermat's Last Theorem, the largest ever machine-checked — davidad · 2026-09-05
- GPU Sandboxes as the Compute Primitive for Recursive Self-Improvement — AAAzzam · 2026-09-05
- AI catastrophe risk is already intolerable, yet the race keeps accelerating — RobbWiller · 2026-09-05
- Moltbook Was Built for Agent Swarms, Yet Zero Consequential Conversations Have Happened — granawkins · 2026-09-05
- OpenAI job listing tracks 'automation of technical staff' amid self-improving AI bets — imjustnewatai · 2026-09-05
- Garrison Lovely's AI-critical book Obsolete lands Sept 29, backed by Acemoglu and Tegmark — GarrisonLovely · 2026-09-05