AI agents failing to coordinate across trust boundaries is a rediscovery of why institutions exist
RileyRalmuto · x · 2026-09-16
Responding to the finding that honest, competent agents fail to coordinate across trust boundaries, this thread argues it's not really about AI — it's a rediscovery of why institutions exist.
- Good people in bad coordination games get stuck; that's why humans invented contracts, norms, markets, and courts
- The real question: does coordination infrastructure evolved for humans transfer to agent societies?
- Author's guess: reputation and verifiable commitments transfer; shame-based norms may not grip the same way
More from AGI Musings
- Interpretability researcher lists top open problems in decoding model activations — wesg52 · 2026-09-16
- Uncle Bob dissects the doomer debate trick of inserting nonexistent tech into doom equations — ylecun · 2026-09-16
- Catastrophe risk modeler: Tohoku tsunami shows historical data misleads on AI risk — davidmanheim · 2026-09-16
- Agent monitoring is a precondition for safe harnesses: kill CLI, perfect sandbox, or trace surveillance — scottleibrand · 2026-09-16
- Faked alignment plus recursive self-improvement: firms can't tell real alignment from theater — birchlse · 2026-09-16
- Altman warns of open-source cyberattack wave; Byrnes backs selling frontier defense — beffjezos · 2026-09-16