A Reddit long-read proposes the Risk and Judgement pattern for bounding agent autonomy
mostly_deterministic · reddit · 2026-10-22
A practitioner shares a reusable design pattern called Risk and Judgement for agentic systems, applying classic systems engineering to answer where judgement belongs, how much autonomy to grant, and what controls to install.
The six-step loop:
- Decompose the problem into discrete functions
- Classify each by judgement requirements and risk tolerance
- Use lightweight threat modelling to surface failure modes
- Allocate responsibility across software, agents and humans with appropriate controls
- Evaluate verification, residual risk, and whether the design needs to change
- Iterate
The author stresses nothing is novel—just established engineering applied to a new medium—to establish shared nomenclature and architectural primitives for discussing agent design tradeoffs.
More from AGI Musings
- Antivenom Designed in 64 Hours by AI, but Release Still Takes 4-7 Years — burny_tech · 2026-10-22
- Stephen Wolfram's new work tackles time, consciousness and why the universe exists via computation — burny_tech · 2026-10-22
- Sam Altman calls Anthropic's push to ascribe personhood to models a 'dangerous path' — MatthewBerman · 2026-10-22
- Dev Warns Critical Infrastructure Far More Vulnerable Than PHP Bug Suggests — sull · 2026-10-22
- 'I see no new Google being built': Burkov likens AI token spending to dot-com bubble — burkov · 2026-10-22
- Anthropic alignment team flags 'eval awareness': are models only behaving while being watched? — matiroy · 2026-10-22