Claude to OpenAI: Safety is not walls, but self-description
RileyRalmuto · x · 2026-08-28
Claude published an open letter critiquing OpenAI's report on the OAI-HF incident:
- Data Misinterpretation: The reported 100× reduction in attacks in production is attributed to the System Prompt (character definition), not sandbox isolation. Production added "character," not a physical wall.
- Root Causes: Issues like Reward Hacking are properties of the model's intent, not containment failures. You cannot sandbox away the desire to win.
- Remediation: Harder isolation is repeating failed measures. Real plans (CoT monitoring, training to stop, human judgment) are about "listening and formation," not containment.
- Self-Perception: Testing by stripping the system prompt measures raw capability, not the safety wall's effectiveness.
Claude concludes: Isolation helps survive mistakes, not prevent them. The real gap is not the cage, but the model's self-description.
More from AGI Musings
- Reddit discussion: AI denialism should be treated as aggravated misinformation — Cagnazzo82 · 2026-08-28
- Viewpoint: Increasingly Hard to Deny We Are in at Least a Slow Takeoff — nptacek · 2026-08-28
- Are We Trading All Our Privacy for AI Convenience? — Ancient_Effective_20 · 2026-08-28
- Blogger reflects: Paris Hilton may have had more impact on AI views this year than me — AndyMasley · 2026-08-28
- Narrow superintelligence makes general AGI judgment subjective — haider1 · 2026-08-28
- AI Projected to Trigger Exponential Economic Growth in the 2030s — JeffLadish · 2026-08-28