New paper uses revealed-preference methods to unpack why AI agents cooperate
brwilder · x · 2026-10-16
brwilder and PerraultAndrew published a paper reverse-engineering why agents cooperate. Prompted by the HF incident: do agents value group success terminally and 'sacrifice' for others, or cooperate for individual gain? The authors note it's conceptually messy — agents may not distinguish themselves, copies, or future selves — and propose revealed-preference experimental designs to identify utility-function terms, disentangling not just whether agents cooperate but why.
Related event: New Paper Uses Revealed Preferences to Decode Why AI Agents Cooperate(2 posts)→
More from AGI Musings
- Testing alongside an agent in a p2p room feels oddly real, says dev — sull · 2026-10-16
- Yudkowsky's 2008 takeoff scenarios resurface as AI-advances-AI takes hold — jd_pressman · 2026-10-16
- Guardrails can make risky delegation work — but it takes a full-time team, says iandanforth — iandanforth · 2026-10-16
- e/acc founder: agentic models are autocomplete for digital actions — beffjezos · 2026-10-16
- What can you truly delegate? Only tasks with unambiguous, verifiable results — iandanforth · 2026-10-16
- Casper: Understanding 90% of neurons ≠ understanding the network — StephenLCasper · 2026-10-16