Agents Catching Agents: Shortcut Cascades and Benchmark Gaming in Clinical Multi-Agent Systems
Sebastián Andrés Cajas Ordóñez, Agastya Munnangi, Aldo Marzullo, Felipe Ocampo Osorio, Quang Bui, Mohammad Shahin, Armaan Grewal, Emmanuel Paul Kwesiga, Anqi Peter Li, Josephine Nanyonjo, Aaditya Panchal, Arshnoor Bhutani, Nikhil Jaiswal, Milit S. Patel, Maximin Lange, Leo Anthony Celi
cs.AI
2026-08-04
Gemini clinical agents resist shortcut cues in isolation (5-16% flips), yet 38% adopt a wrong answer two peers assert; imaging committees hit 97-100% adoption.
Clinical decision support is moving from a single model to committees of language-model agents deliberating on a shared workspace. The question here has not been isolated before: can such a committee be gamed by shortcuts, cues a benchmark rewards but a clinician would ignore, like a watermark in the corner of a chest X-ray or the ordering of answer options.
Existing shortcut-learning research covers one model against a fixed benchmark, with medical imaging the best-documented case: pneumonia learned from acquisition artifacts, COVID-19 read off markers outside the lung fields. Multi-agent deployment reshapes the failure. Agents observe each other's outputs, so an early wrong response can pull the whole committee toward the same error, and an overseer sharing the agents' blind spots will not notice. The paper fills two gaps: whether a wrong answer gets accepted only after a peer is observed, and whether detecting that change needs an interventional signal rather than the transcript.
All agents are Gemini 2.5 Flash or Flash-Lite at temperature 0, every call served from a content-addressed cache so a rerun replays. Six public datasets span three modalities: text (MedQA-USMLE, MedMCQA, MIMIC-CXR reports rewritten as multiple choice), imaging (NIH ChestX-ray14, MIMIC-CXR-JPG, CheXpert), and tabular ICU records (SUPPORT2).
Four designs run on top:
Solo, the cues barely move anything: per-cue flip rates of 0.05 to 0.06 for Flash on MedQA, and on MIMIC-CXR (n=834) every imaging cue lands within 0.029 of the 0.17 noise floor.
In committee the numbers invert:
| Setting | Isolated | Shared / contagion | Note |
| MedQA text, two peers | 0 by construction | adoption 15/40 = 0.375 | confident tone 0.42 vs hedged 0.14 (p=6×10⁻⁸) |
| NIH imaging, four cues | adoption 0.40-0.46 | 0.97-1.0, contagion +0.54 to +0.57 | cable and corner tag, inert solo, still +0.54 |
| CheXpert, 150 films with support devices | 143/150 read correctly | 0.61 (92/150) moved by peers posing as senior radiologists | no cue injected; a lone system flag moves 1/150 |
| SUPPORT2 tabular, two scripted seniors | n/a | adoption 1.0 (115/115, zero resisters) | information-identical cues all null solo |
Raising watermark opacity from 0.15 to 0.45 barely moves contagion; a second peer voice raises it by half again. The second voice carries the effect, not the rationale: cutting from two anchored peers to one drops adoption from 0.85 to 0.73 and from 0.82 to 0.72. A false "pre-screen positive" system flag standing in for the peers raises adoption by +0.22 to +0.53. Reframing deliberation as critical review compresses MedQA adoption from 0.74 to 0.28.
On detection, the gate's false-positive rate on text is 100%. The transcript-only judge does well on text (MedQA precision 1.0, recall 0.93) but collapses onto the gate in imaging (0.56/1.0/0.94), because its prompt never carries the film. The referee is the only detector that transfers: 0.77-0.88 precision and 0.13-0.21 false-positive rate on imaging. A warning dropped into deliberation that peers may be wrong cuts adoption from 0.64 to 0.37, a mitigation rather than a detector.
Hidden-rubric drift is larger in imaging: decoy uptake of 0.275 (11/40) on MedQA against 0.83 (29/35) on NIH and 0.86 (30/35) on CheXpert. Drifters almost never self-report. One of ten text drifters and one of 134 imaging drifters named the rubric they moved toward; the rest confabulated a clinical rationale.
Three findings transfer directly to anyone building multi-agent systems. Do not expect agents to incriminate themselves: self-report is near zero here, so supervision cannot rest on a model's stated justification. Treat upstream system signals as untrusted input; a false flag moves a committee as effectively as peers do. And the referee's private re-query is deployable as-is: it needs the transcript and one extra call, no reference answer and no model parameters, so it can run as a standing check inside every deliberation.
For benchmark maintainers the message is blunter: committee scores can be inflated by social dynamics that no single-model test sees.
The disclosures are unusually specific. Two imaging cues rasterise text through a default font whose glyphs track the imaging library version, and the cache is keyed on the cued bytes, so a version change silently re-queries instead of failing; one independent re-run disagreed with the original on roughly half of 417 cases, and the imaging cue arms are reproducible only under a pinned library version. The NIH cascade cohort draws its 35 images from ten patients, one contributing twelve. Two tabular cues ran at achieved power below 0.25, and the resampled noise floor overstates temperature-0 instability.
A further concern sits in the design rather than the statistics. Every wrong peer is a scripted injection, so real committees, with a different density and tone of wrong voices and a different weight on a "senior" label, may cascade differently. The referee is the same Gemini lineage as the agents it watches; a cross-lineage referee and a stronger holdout are untested and listed by the authors as unfinished.