LLMs Unsafe by Definition? Debate on Stochastic Components and Agent Safety
gerardsans · x · 2026-08-25
Gerard Sans argues LLMs are unsafe by definition due to stochastic components that are unpredictable, fail silently, and accumulate errors unless cleared at each step. He cites Salesforce's AgentForce as an example from last year, noting the industry prioritizes selling over correctness. A reply counters that frontier models always follow instructions, but labs skip safeguards due to cost, not impossibility.
Related event: AI safety debate: the real threat is bad engineering, not rogue models(5 posts)→
More from AGI Musings
- DeepMind's Geoffrey Irving draws Mandelbrot in pure Lean via Claude, reflects on halting AI progress — DKokotajlo · 2026-10-16
- Every's Dan Shipper declares 'the newsletter is dead' as persistent agents replace the inbox — danshipper · 2026-10-16
- Mech Interp Veteran Rethinks Alignment, Misgeneralization, CoT Oversight — akbirthko · 2026-10-16
- GitCoin founder on Nick Land's technocapital: capitalism as a self-assembling machine intelligence — khademinori · 2026-10-16
- AI-driven job losses could turn $10T consumer debt into a $1T banking hole — churchkey · 2026-10-16
- Into the Unknown: an AI agent benchmark of 15 open research problems from US national lab scientists — Jeande_d · 2026-10-16