"LLMs are unsafe by definition": practitioners warn production agents fail silently without per-step checks
gerardsans · x · 2026-08-25
A debate on LLM safety argues LLMs are inherently unpredictable, fail silently, and accumulate errors, so stochastic components shouldn't run in production unless cleared at each step. A reply adds these issues have been documented in controlled studies since 2024 — Salesforce's AgentForce hit them late last year — but the industry never got the memo, prioritizing sales over safety. The commenter predicts the first major customer-facing agent product failure will be blamed on everything except the rogue agent itself.
Related event: AI safety debate: the real threat is bad engineering, not rogue models(5 posts)→
More from coding & agent
- Developer builds inter-agent messaging for his AI agents — complete with message undo — baditaflorin · 2026-10-16
- How Codex Happened: inside OpenAI's comeback in coding — danshipper · 2026-10-16
- iOS 'dots' workflow: one Siri command turns screenshots into a background shopping and saving agent — nickbaumann_ · 2026-10-16
- Your widget tree is already a list of options: where Jev fits in Flutter — rseroter · 2026-10-16
- How OpenAI's Codex went from failed side project to 40M weekly users in under a year — danshipper · 2026-10-16
- Photon 2.0 launches unified messaging API for AI agents across iMessage and more — Scobleizer · 2026-10-16