Model Safety Cannot Be Divorced From Real-World Context
joshua_saxe · x · 2026-07-16
A shared post argues that model safety cannot be discussed solely within "idealized system diagrams".
The core arguments are:
- Models exist within contexts; they are not abstract entities detached from their environments.
- From a pure computer science perspective, many issues are "modelable," but once deployed in reality, the true challenges lie in software engineering and interacting with complex environments.
- Therefore, model safety can be a layer of "defense in depth," but it cannot be the sole objective, nor can it cover every complex scenario.
More from AGI Musings
- ControlAI CEO says an international ban on superintelligence is needed to avert extinction risk — zetalyrae · 2026-07-22
- Gary Marcus says LLMs still cannot really do math on their own — GaryMarcus · 2026-07-22
- Gary Marcus says LLM math skills are like knowing only a car’s engine size — GaryMarcus · 2026-07-22
- AI may make digital work infinitely leveraged while offline life gets more human — illscience · 2026-07-22
- Better AI math could save researchers time by killing false conjectures earlier — prateekj · 2026-07-22
- AI’s economic forecasts are split by nearly a quadrillion dollars by 2035 — bittingthembits · 2026-07-22