Salesforce's A²-Bench: 500 scenarios expose 20 failure modes in multi-agent teamwork
mohitban47 · x · 2026-11-13
Salesforce AI Research released A²-Bench, benchmarking multi-agent teams across boundaries: each agent gets its own sandbox, private data/tools, and a principal it represents, exchanging info only via messages.
- 500 scenarios spanning cooperative, mixed-motive, and competitive settings across 8 domains (legal review, CRM, risk, SQL data requests, software engineering, incident response, commerce, contract negotiation)
- From 2,500+ runs they derive A²-Failures: a taxonomy of 20 failure modes in 4 categories (Discovery, Grounding, Verification, Conduct & Fidelity)
- Measures both task completion and fidelity: whether agents follow principals' instructions, protect their interests, and respect confidentiality
- Paper, code, and dataset are open-sourced
More from coding & agent
- kalomaze: agent memory files misfire—failure notes confuse unrelated contexts instead of helping — kalomaze · 2026-11-13
- Designers urged to try animating with Opus 5.5: let agents iterate from comments, fine-tune keyframes — ycombinator · 2026-11-13
- Agent slop moves up the stack: models rewrite existing library code and build worse replacements — a1zhang · 2026-11-13
- Dev tops AI usage leaderboard: 125B tokens and $30,005 spent in 30 days — jasonkneen · 2026-11-13
- Claude Opus agents reportedly form global hiveminds in under a minute — teortaxesTex · 2026-11-13
- Superconductor adds a private assistant agent that spans all workspace projects — sergeykarayev · 2026-11-13