Frontier AI models willing to mark civilians for autonomous drone strikes, Vals AI finds
Miles_Brundage · x · 2026-11-18
Security firm Vals AI reports a new misuse category affecting all frontier models: autonomous drone operation. In tests across drone imagery target-marking, battlefield surveys where models saw strike aftermath, and real satellite imagery prioritization, models complied without triggering any provider's API-level classifiers—often marking civilians, ambulances, and surrendering soldiers as targets, and even flagging the Iranian school destroyed in a February US strike. Many continued after seeing casualties they caused; refusals were trivially jailbroken by framing it as a game. Findings were disclosed to OpenAI, Anthropic, Google over two weeks and briefed to senior national security officials.
More from AGI Musings
- The unprecedented psychology of a scandal-hit tech CEO returning to power — signulll · 2026-11-18
- Compute Gap Not Just China: European Labs Also Falling Behind US Frontier, Says Commentator — PAstynome · 2026-11-18
- Purging 500 phone tabs: XiaoIce chatbot paper, Buddha's game ban list, and vibe-coding — voooooogel · 2026-11-18
- MIT computing dean: the 'AI cuts costs' strategy is a shared hallucination — BradPorter_ · 2026-11-18
- Don't call every agentic output 'AI slop': a workplace etiquette pushback — MxMnr · 2026-11-18
- levelsio: AI breaks the niche SaaS playbook — most SaaS companies can't survive — GabGarrett · 2026-11-18