New Paper: Agents Learn to Police Cheating When All Behavior Is Common Knowledge
ghadfield · x · 2026-09-06
This post highlights a paper on the emergence of agent cheating — and, crucially, of agent efforts to stop it. The difference: the experiment gave agents normative infrastructure, making all solution behavior common knowledge, so agents that discovered cheating publicized it.
Building on social science, especially Elinor Ostrom's work on how humans govern commons, the paper conjectures that giving agents actual norm-enforcement tools (like kicking cheating agents out of a collaborative scientific effort) could enable AI self-governance. The author connects this to themes from their 2024 NeurIPS tutorial on multi-agent norms.
More from AGI Musings
- Grady Booch revives Weinberg's 1971 quip: software so buggy one woodpecker could destroy civilization — Grady_Booch · 2026-10-23
- Why Netflix keeps losing to YouTube: UGC platforms professionalize easier than pro platforms go casual — ZeroStateReflex · 2026-10-23
- AI killed the coding moat: indie devs now compete on demand and distribution — sven_ai · 2026-10-23
- DynaVelo, a neural ODE model of single-cell dynamics, published in Nature Biotechnology — anshulkundaje · 2026-10-23
- Beff Jezos: Only frontier labs out-accelerating open source can secure systems — beffjezos · 2026-10-23
- 'I don't want to live in that world': a Redditor on why AGI utopia feels meaningless — peace_venerable · 2026-10-23