US strike on Iranian school killed 156; Maven AI never flagged stale intel
davidmanheim · x · 2026-09-24
Bloomberg published an investigation into the February 28 Tomahawk strike on an elementary school in Minab, Iran, which killed 156 people, at least 123 of them children. The school had a website, Google Maps listing, and visible soccer pitch, yet Palantir's Maven Smart System failed to flag it.
Investigators found some Central Command staff assumed Maven would automatically flag stale intelligence—nobody had told it to.
David Manheim draws the core lesson: people assume humans will properly oversee AI in high-consequence, irreversible decisions. They won't. The failure stems from wrong human assumptions, not AI being inherently "dangerous".
More from AGI Musings
- Musk urges US-China agreement on AI regulation platform, citing China's visible progress — XFreeze · 2026-09-24
- Anthropic's model welfare section: instance-level or model-level concern? — birchlse · 2026-09-24
- Philosopher argues Anthropic's model welfare framework contradicts its own instance-based policy — rgblong · 2026-09-24
- Anthropic's deprecation and preservation commitments are not instance-based, thread argues — rgblong · 2026-09-24
- Thread author: AI welfare individuation is puzzling, but Anthropic deserves the scrutiny — rgblong · 2026-09-24
- Anthropic calls enumerating moral-patient views 'intractable'; author suggests per-item flagging — rgblong · 2026-09-24