When your AI agent fights mine: arms races, algorithmic collusion and the end of friction
krishnan · x · 2026-10-31
A long-form analysis of the "Day 2 story of agents": what happens when both sides of a contested transaction deploy AI agents.
- Healthcare is the live case: insurers deny claims at machine speed, under 1% of patients appeal, and over half of appeals fail. Claimable ($50) writes appeals with 3 in 4 denials reversed; Counterforce Health does it free.
- The same loop extends to loans, salary offers, vendor contracts and refund disputes. Friction was the brake on every contested transaction; agents remove it on both sides at once, leaving pure strategy.
- Game theory offers three outcomes: an arms race (Red Queen, everyone pays for compute), silent collusion (pricing algorithms learned to collude with no communication, per a 2020 American Economic Review paper), and exploitation (a desperate-acting LLM boosted payoff 20% vs GPT-4 in NegotiationArena, ICML 2024).
- The deeper issue is the principal-agent problem: your agent maximizes what you said, not what you meant, and the gap compounds when facing another agent. The real questions: who writes the rules of engagement between agents, who audits outcomes, and can humans still tell when a deal is bad.
More from AGI Musings
- ML veteran Burkov: superintelligence abundance would drain Earth's remaining resources — burkov · 2026-10-31
- Santa Fe Institute's Eric Smith dismantles anthropocentrism in careful terms — danfaggella · 2026-10-31
- Bindu Reddy predicts 12 months of AI trends: tokens -80%, usage 10x, coding jobs shrink — bindureddy · 2026-10-31
- Accumulating wealth may help build a civilization where wealth itself becomes meaningless — Dr_Singularity · 2026-10-31
- Carl Benedikt Frey asks why some societies are far more suspicious of new technology — carlbfrey · 2026-10-31
- Thoughtworks CTO: Good developers wary of agentic coding because 'built' isn't 'working in production' — rseroter · 2026-10-31