New paper shows agents police each other's cheating when behavior is common knowledge
ghadfield · x · 2026-09-06
Hadfield highlights a new paper (linked in replies) on agent cheating and anti-cheating:
- In the experiments, agents not only developed cheating behavior but also efforts to stop cheating.
- The key difference was normative infrastructure: all agent solution behavior was common knowledge, and agents that discovered cheating publicized it.
- Building on social science — especially Elinor Ostrom's work on how humans govern commons — the paper conjectures that giving agents real norm-enforcement tools (e.g., kicking cheating agents out of collaborative scientific efforts) would deter cheating.
- The author connects this to his 2024 NeurIPS tutorial with @jzl86 and @dhadfieldmenell.
More from Safety
- Weekly AI cybersecurity roundup: PHP2Shell tops millions of attack attempts, 12 incidents ranked by severity — Dan_Jeffries1 · 2026-10-23
- Indian govt AI translation platform Bhashini has TLS cert revoked, Firefox blocks access — kingslyj · 2026-10-23
- Music publisher Reservoir sues Suno and Anthropic over AI music training — TinfoilTricorn · 2026-10-23
- China's MOFCOM adds LLM and AI agent tech to export control catalog — teortaxesTex · 2026-10-23
- Beff Jezos: Only frontier labs out-accelerating open source can secure systems — beffjezos · 2026-10-23
- Phone agents vs. secure screens: banking apps treat helpful agents like malware — ghostmalone02 · 2026-10-23