Google's CLASHarness benchmarks multi-agent conflicts in computer use, finds frontier VLMs struggle
google · hf · 2026-10-16
Google researchers, with Emory, USC and other universities, introduce CLASHarness, a controllable and verifiable pipeline for generating multi-agent conflict benchmarks in computer-use tasks.
Motivation: existing benchmarks assume cooperative, consistent environments, while real-world collaboration involves outdated reports, contradictory instructions and competition over shared files and permissions.
Manual task authoring doesn't scale and unconstrained LLM generation suffers from narrative drift, answer leakage, and unsatisfiable rubrics. CLASHarness addresses this with a controllable generation framework.
Findings: frontier vision-language models struggle with verifying ground truth, resolving conflicts, and coordinating shared resources.
More from coding & agent
- Running a multi-model UI design arena: Paseo orchestrates four design drafts per model — vista8 · 2026-10-16
- Cortex A/B-tests coding-agent skills and rules by replaying developer corrections before keeping them — Samuele Giampieri · 2026-10-16
- Tool calls are the hidden bottleneck killing AI coding speed — Crabbox and 5 fixes — steipete · 2026-10-16
- Agentic sandbox WorldBox doubles GPT-6 Astra's MazeBench score, 23 to 61 gems — ZhitingHu · 2026-10-16
- Two prompts: Gemini 4 Argon turns a single-player game into full multiplayer — chongdashu · 2026-10-16
- Anthropic and Gray Swan launch $300,000 indirect prompt injection red-teaming contest — bcherny · 2026-10-16