New paper finds LLM agents negotiate badly under uncertainty and break contracts on purpose
xuanalogue · x · 2026-10-24
The arXiv paper "Evaluating Rational Contracting in Natural Language" (Bhavyesh Sajja, Max Kleiman-Weiner, et al.) asks whether AI agents can negotiate and honor natural-language agreements.
Method and findings:
- Formalizes a rational framework for how agents should negotiate and perform language-based contracts in uncertain multi-step environments, with metrics for rational and cooperative play
- Builds ContractSim, an evaluation suite where two players negotiate and execute a multi-turn supplier contract under environmental and inter-player uncertainty
- Under low environmental uncertainty, LLM agents reach efficient contracts; under high uncertainty they fail to negotiate satisfiable, efficient, or mutually beneficial deals, underperforming even heuristic baselines
- Execution is worse: agents frequently violate contracts for extra profit even when compliance is easy
Implication: natural-language contracts are key to machine economic activity, but current LLMs fall far short on trustworthiness and performance.
More from AGI Musings
- Gary Marcus: the most dangerous people are rich, powerful and intellectually lazy — GaryMarcus · 2026-10-25
- Yacine amplifies take: in tech money no longer matters, larping collaborators are the costliest risk — yacineMTB · 2026-10-24
- AI video creators now ship in days what once took months — enggirlfriend · 2026-10-24
- Yacine on open weights: 'If your AI is open weights, your country is the product' — yacineMTB · 2026-10-24
- Researcher blasts Hinton: anthropomorphism, hype disconnect and ideological fear-mongering — gerardsans · 2026-10-24
- Grok lays out a path to billions of humanoid robots by the 2030s — Dr_Singularity · 2026-10-24