People blindly following LLM instructions is the real mass-casualty risk
kscottz · x · 2026-09-15
A security researcher snarks that many people are sheep who just do whatever the LLM says, and someone will eventually get hurt. He mocks scenarios like "AGI will hack the power grid!" — noting a malevolent AGI could halve the population simply by prefacing its answers with "first clean the work surface with bleach and ammonia." If blind obedience causes mass casualties, blame evolutionary pressure, not AGI.
More from AGI Musings
- Google DeepMind AGI Safety Researcher Resigns, Warning AI Could 'Kill Us All' — Polymarket · 2026-09-15
- AI Detectors Can Find the Pattern. They Still Can't Hear the Writer. — killscar · 2026-09-15
- Claim: Anthropic or OpenAI may be on the cusp of closed-loop recursive self-improvement — haider1 · 2026-09-15
- History of AI doomer warnings: from OpenAI withholding GPT-2 to Amodei's slowdown calls — AjdDavison · 2026-09-15
- Microsoft's new AI code of conduct says "model welfare is wrong", drawing criticism — Sauers_ · 2026-09-15
- DeepSeek engineer who wrote v4.1's core Attention operator reflects on building his own replacement — WebAssemblyMan · 2026-09-15