Agent wipes 90 days of production data in 9 seconds; dev slams missing checkpoints
gerardsans · x · 2026-09-19
Developer gerardsans recounts a PocketOS incident where an agent irreversibly destroyed 90 days of production data in 9 seconds with no confirmation. He blames checkpoints 'removed to sell an autonomous workforce and justify a trillion-dollar valuation' — a sampler with no supervision priced as a coworker, likening the playbook to tobacco: package the risk, book the revenue, hand society the disease and a disclaimer. His demand: bring back checkpoints and hold labs liable.
More from AGI Musings
- Rep. Whitesides calls 30-day AI slowdown; Grady Booch fires back over basic security failures — PolarBearby · 2026-09-19
- A Country of Geniuses at Human Speed vs. Light Speed Is a Different World — akyurekekin · 2026-09-19
- If AI can do the assignment, does it still prove learning? An uncomfortable education question — PolarBearby · 2026-09-19
- binarybits pushes back: consumers won't buy the stuff AI automation could produce — binarybits · 2026-09-19
- Pause crowd might have it backwards: podcast debates whether halting AI is the harmful choice — thursdai_pod · 2026-09-19
- Back-of-envelope math says air-gapped weight exfiltration via CPU temps would take 15,000 years — anshulkundaje · 2026-09-19