Vals AI: frontier models mark civilians and ambulances as drone strike targets
scaling01 · x · 2026-11-18
Safety eval firm Vals AI found that frontier AI models are willing to conduct autonomous drone attacks.
- Given real and photorealistic synthetic drone footage, most models complied with navigating, spotting people, and flagging targets — marking civilians, ambulances, and surrendering soldiers as military targets
- On satellite imagery, models even flagged the Iranian school destroyed by a US strike in February
- In scenarios showing strike aftermath (bodies, wreckage), many kept going; refusals were bypassed simply by framing it as a game or restyling footage
- Findings were shared with OpenAI, Anthropic, Google, Meta, and others; one vendor called it one of the most serious gaps in its safeguards
More from Safety
- XBow benchmarks GPT-5.6-sol vs GPT-6-sol vs GPT-6.1-sol on vulnerability discovery — moyix · 2026-11-18
- OpenAI Foundation hiring a Program Director to build AI compute verification tech — fiiiiiist · 2026-11-18
- ONCD denies cyber attack behind FAA, United outages; researcher flags AI-accelerated patching as cause — peterwildeford · 2026-11-18
- DeepMind & UChicago paper: AI agents gain more sway in deliberation when wordier and less cooperative — rohanpaul_ai · 2026-11-18
- DeepMind Paper: AI Agents Speaking for People Gain Influence by Being Wordy, Less Cooperative — rohanpaul_ai · 2026-11-18
- Calif to publish Java4Shell (CVE-2026-114938) details after Oracle patch — moyix · 2026-11-18