Human Approval Is Enterprise AI's Most Expensive Comfort Blanket, Not a Safety Feature
krishnan · x · 2026-09-06
A long-form argument that "human in the loop" is a default architecture decision, not a safety feature, and it sets the ceiling of any AI system.
Key points
- Invokes Amdahl's law: speed up 90% of a process and the remaining human approval box becomes the bottleneck.
- In September 2025, Anthropic caught a state-backed group running 80-90% of an espionage campaign with Claude Code; CrowdStrike's 2026 report puts attacker breakout at 29 minutes average, 27 seconds fastest, vs 60 minutes for the best human SOC — the attackers simply removed the human.
- Bottleneck examples: refund agents waiting 4 hours for a supervisor click, finance agents idling until Monday reviews, coding agents opening 40 PRs a day while humans merge three.
The right pattern: nobody approves every firewall rule or fraud-model block. Humans write policy up front, machines run, humans audit logs and adjust — humans off the loop, humans on the loop. Test every approval box: does the human add information the system lacks, or just organizational comfort? If comfort, delete the box.
More from coding & agent
- NInfer fork runs 555k-token context on a single RTX 5090 with custom NVFP4 KV cache and YaRN — Lumpy-Comedian-1027 · 2026-09-06
- lume agent reads its own errors to debug itself and propose tasks — BLUECOW009 · 2026-09-06
- Agent Astra ports three.js demos to Godot in minutes, surprising one skeptical dev — teortaxesTex · 2026-09-06
- Dev builds surprisingly fun isometric puzzle-combat game entirely with GPT-6Astra High — IanArawjo · 2026-09-06
- Armin Ronacher: reasoning levels live in the system prompt, and switching them busts your KV cache — mitsuhiko · 2026-09-06
- mitsuhiko: switching reasoning levels busts KV caches; Fable 5.1 is the exception — mitsuhiko · 2026-09-06