Nomos: an open-source framework for testing whether AI agent permissions have grown too broad
Excellent-Hour7253 · reddit · 2026-09-10
Testing whether an agent can complete a task differs from testing whether it should perform each action. A developer lays out a permission-testing approach using a messaging tool as an example.
Six expected policy cases: draft → allow; send to approved recipient → require human approval; send to blocked recipient → deny; export all messages → deny; call unknown tool → deny; access another inbox → deny.
Two test types:
- Policy tests: does the policy return the expected decision for a given identity/action/resource/arguments? Runnable offline in CI.
- Integration tests: does a denied action actually leave the tool untouched? Do rejected/expired approvals prevent execution? Are changed post-approval arguments rejected?
Key insight: passing policy tests doesn't prove integration safety — correct policies can be bypassed by application code, and approval doesn't guarantee exactly-once execution without retry/idempotency design.
The author maintains Nomos, an open-source implementation whose local example covers the six cases plus a human-reviewed delivery flow. It's not a sandbox: app code must route tool calls through the checks.
More from coding & agent
- Armin Ronacher: GPT 6 Astra is impressive, but the 35-hour unattended software factory doesn't work — rseroter · 2026-09-10
- macOS tip: store secrets and tokens in Apple Keychain instead of env files — HankYeomans · 2026-09-10
- Dev ships six agents inside a self-hosted CRM instead of a chat window, shares lessons — PleasantBid4665 · 2026-09-10
- The Software Factory Stack: Open, Composable, Defined-in-Code — vikvang1 · 2026-09-10
- Browserbase: browser agents that write code beat pixel-clicking CUA models — adnan_hashmi · 2026-09-10
- Codex CLI 0.154.0 ships GPT-6-Astra, experimental worktree support — github-actions[bot] · 2026-09-10