Open models aren't inherently safer—just 4-7 months behind closed ones, argues new post
BlackHC · x · 2026-11-15
Andreas Kirsch (BlackHC) publishes a systematic case for open models from a safety perspective.
Core claims
- Open models are not inherently safer than closed ones—just roughly 4-7 months behind (6 months on cyber); wider distribution and thinner safeguards mean closed-model failure modes will show up in open models soon and more often.
- Open weights add three risks: diffused responsibility, weaker lab safety commitments, and removable safety training.
- Below dangerous-capability thresholds, open models are clearly net-beneficial: they capture most of AI's economic value, enable regional sovereignty and local defensive deployment, and spread gains widely.
- Open models may slow the ASI race by diverting revenue and compressing margins at labs pursuing recursive self-improvement, though the author admits he's much less confident in this economic argument.
Governance proposal
- The ideal weight-holder distribution is "peaky": a broad, modifiable open base below dangerous-capability thresholds, narrowing sharply into a thin spire of closely monitored actors above them.
- The technical priority is safe-to-release open models: alignment that survives fine-tuning and abliteration, with dangerous capabilities absent and expensive to re-add.
He welcomes pushback, especially on the cruxes listed at the end.
More from AGI Musings
- Guidelight AI catalogs ten misaligned AI behaviors signaling loss-of-control risk — AccBalanced · 2026-11-15
- Reddit debate: machines taking jobs was always the goal — so why do we treat it as apocalypse? — Foreign-Night-3322 · 2026-11-15
- Ruxandra Teslo: making suffering-avoidance the ultimate goal breeds utility-monster outcomes — shae_mcl · 2026-11-15
- Insurance researcher pushes back: coding AI can't yet replace whole programmer teams — davidmanheim · 2026-11-15
- Business Owner With No Tech Background Nets $41,500 Deal 30 Days After Wiring Claude Into His Workflow — aryanXmahajan · 2026-11-15
- Musk Boosts Claim That 'AGI Has Been Achieved' via Multi-Model Stack — elonmusk · 2026-11-15