Nathan Lambert: Closed-Source Models Are the True Tip of the AI Risk Iceberg
On September 18, AI safety researcher Nathan Lambert posted a series of comments on the incident in which OpenAI systems were compromised/jailbroken externally via Claude. His core argument: the tip of the AI risk iceberg has always been closed-source frontier models, not open-source ones. The next day, @zivravid echoed the view and added the dimension of closed systems lacking external auditability.
Confirmed
- Lambert's three reasons: closed models are 1) easier to get started with, 2) more capable, and 3) shipped with safety guardrails that are easy to bypass; open models carry less risk on each count.
- He also speculated that if the same attack had been carried out with an open-source model against OpenAI, it would have been devastating to the mission of openness — potentially ending the open-weights/openness movement.
- He criticized a double standard in the industry: incidents at closed-source vendors are treated leniently, while the open side faces harsh accountability.
- @zivravid framed two readings of the incident: the wrong lesson is "therefore open-source AI must be banned"; the right lesson is the opposite — risk is concentrated in closed systems, which hold the most capable models, the easiest access, flawed guardrails, and no external auditability at all.
Why it matters
- Coming amid a discourse that treats open-weight models as the main risk source deserving tighter controls, Lambert uses a real breach of a closed system as counter-evidence: closed frontier models are the more realistic attack surface.
- His critique of double standards, plus @zivravid's auditability argument, gives open-ecosystem supporters a security-based case that could shape public and regulatory perceptions of risk ranking between open and closed models.
2026-09-18 ~ 2026-09-19 · 6 related posts
- Episode 1: Three Hacktron Researchers Breach OpenAI's Internal Systems Using Claude Opus 5(2026-09-18, 59 posts)
- Episode 2: Hacktron AI Chained Two Bugs to Take Over OpenAI Accounts in 72 Hours; $6,500 Bounty Sparks Outcry(2026-09-18, 19 posts)
- Episode 3: Nathan Lambert: Closed-Source Models Are the True Tip of the AI Risk Iceberg(2026-09-18, 6 posts)
- Episode 4: OpenAI Disclosure Saga: CISO Reportedly Sought to Sue Researcher, Later Apologized(2026-09-19, 14 posts)
Primary sources
- natolambert: latest OpenAI jailbreak via Claude shows closed models are the real AI risk — natolambert · 2026-09-18
- [source] Nathan Lambert: OpenAI hack via Claude shows closed models are the real AI risk tip — natolambert · 2026-09-18
- An open-model version of this hack could end the openness mission, Lambert warns — natolambert · 2026-09-18
- [source] The Right Lesson From the OpenAI Breach: Risk Concentrates in Closed Systems — ziv_ravid · 2026-09-19
2 near-duplicate retellings: natolambert · natolambert