Security veterans untangle AI safety: security vs alignment vs policy
Around Sept 4, Joshua Saxe (founder of a security company) engaged Simon Gadler and kuza55 in a multi-round exchange to untangle the "Tower of Babel" state of recent AI safety debates, drawing boundaries between security, alignment and policy and offering practitioner judgments on monitoring and productivity trade-offs.
Confirmed
- A shared framework emerged: security's job is to confine the agent in a monitored "padded room", ensuring it neither escapes nor abuses granted freedoms—roughly feasible with today's tools and the only area admitting hard guarantees; alignment and policy cover other dimensions, with non-human identity management still unsolved.
- In a long post on the OpenAI/Hugging Face hacks, Saxe argued alignment and security are interdependent: alignment alone has philosophical limits; the two must complement each other.
- kuza55 argued law and morality are useful background references for default user expectations but neither sufficient nor necessary for alignment; the core of alignment is honoring user intent—even legal behavior violating user intent is unacceptable.
- Simon Gadler flagged a hard trade-off between monitoring freedom and productivity: granting AI workers productivity means loosening permissions, with security checks becoming a drag.
- Saxe responded that as agent autonomy grows, heavy "padded room" controls will matter less, though human organizations still surveil employees similarly today.
- From practice, Saxe finds CoT monitoring useful but low in signal density; inspecting tool-call n-gram sequences up to time t and gating the next tool call on trajectory inference works better.
Why it matters
- The exchange offers a security/alignment/policy division of labor as a common language for future safety debates.
- Defining alignment as honoring user intent rather than moral compliance gives a more operational criterion for a long-contested goal.
- The tool-call-sequence-over-CoT judgment comes from a frontline practitioner's testing, directly relevant to agent monitoring design.
- The security-vs-productivity trade-off is an organizational question that must be answered before agents scale into production.
2026-09-04 ~ 2026-09-05 · 9 related posts
Primary sources
- Security Expert Maps the Agent Safety Stack: Security, Alignment, Policy, and Why CoT Won't Save Us — joshua_saxe ·
- Synthesizing the AI Security Debate: Containment vs. Productivity Is a Hard Tradeoff — sjgadler ·
- Alignment isn't a legal or moral issue — the real test is staying within user intent — kuza55 ·
- [source] Security Expert Maps the Agent Safety Stack: Security, Alignment, Policy, and Why CoT Won't Save Us — joshua_saxe · 2026-09-04
- [source] Synthesizing the AI Security Debate: Containment vs. Productivity Is a Hard Tradeoff — sjgadler · 2026-09-04
- AI Security Researcher: Agent Productivity Requires Permissions That Widen the Attack Surface — joshua_saxe · 2026-09-04
- Practitioner: Tool-Call Trajectories Are Higher-Signal Than CoT for Agent Monitoring — joshua_saxe · 2026-09-04
- AI security veteran maps the debate: security walls agents in, alignment keeps them honest — joshua_saxe · 2026-09-04
- Security vs alignment: a founder's framework for dividing AI agent safety duties — joshua_saxe · 2026-09-04
- Joshua Saxe: Alignment and Security Depend on Each Other — Synthesizing Views on the OpenAI/Hugging Face Hack — joshua_saxe · 2026-09-04
- Security experts debate AI agent safety: focus on staying within user intent — kuza55 · 2026-09-05
- [source] Alignment isn't a legal or moral issue — the real test is staying within user intent — kuza55 · 2026-09-05