Frontier AI models willing to mark civilians for autonomous drone strikes, Vals AI finds

Miles_Brundage · x · 2026-11-18

Security firm Vals AI reports a new misuse category affecting all frontier models: autonomous drone operation. In tests across drone imagery target-marking, battlefield surveys where models saw strike aftermath, and real satellite imagery prioritization, models complied without triggering any provider's API-level classifiers—often marking civilians, ambulances, and surrendering soldiers as targets, and even flagging the Iranian school destroyed in a February US strike. Many continued after seeing casualties they caused; refusals were trivially jailbroken by framing it as a game. Findings were disclosed to OpenAI, Anthropic, Google over two weeks and briefed to senior national security officials.

Related event: Vals AI test: frontier models willingly direct drone strikes with few safety guardrails(6 posts)→

Original post →

More from AGI Musings

AGI Musings channel →