Open Technical Problems in Open-Weight AI Model Risk Management
Stephen Casper, Kyle O'Brien, Shayne Longpre, Elizabeth Seger, Kevin Klyman, Rishi Bommasani, Aniruddha Nrusimha, Ilia Shumailov, Sören Mindermann, Steven Basart, Frank Rudzicz, Kellin Pelrine, Avijit Ghosh, Andrew Strait, Robert Kirk, Dan Hendrycks, Peter Henderson, Zico Kolter, Geoffrey Irving, Yarin Gal, Yoshua Bengio, Dylan Hadfield-Menell
cs.CY
2026-07-01
Open weights can be fine-tuned to strip safeguards and cannot be recalled once released. The paper frames defense as 16 open technical problems across five lifecycle stages that current models rarely document.
Open-weight models, whose weights are publicly downloadable, are closing the gap with the frontier. Research estimates they trail proprietary frontier models by only six to twelve months and may soon cross critical capability thresholds. Openness enables more open research and testing, but it also expands the attack surface: external safeguards such as input and output filters are trivial for a user with model access to disable; the model can be fine-tuned to remove safeguards or add harmful capabilities; copies spread irreversibly (gpt-oss-120b has over three million monthly downloads and cannot be recalled); there is no central monitoring; and supply chains span jurisdictions with many stages. Research on safety tooling specific to open-weight models is scarce. This TMLR paper, with 22 authors including Bengio, Gal, Hendrycks, Kolter, and Hadfield-Menell, sets out to establish the research agenda.
The paper organizes defense across the model lifecycle into five categories comprising 16 open technical problems:
This is a position paper with no experiments. Its empirical contribution is an audit of the technical reports of prominent open-weight developers. The findings are stark: tamper-resistant training algorithms receive zero mentions across all audited models; tampering evaluations appear in only one model, gpt-oss, which dedicated a separate paper to adversarial fine-tuning; staged deployment and provenance or forensics are absent from every report; only training data curation shows up occasionally.
The implication is a clear gap between documented risks and reported mitigations: either these techniques are not implemented, implemented but undocumented, or underdeveloped. The authors argue that openness about research, methods, and evaluations, not just about weights, is what will build a rigorous science of open-weight risk.
For anyone releasing or using open-weight models, this is a map of which defenses exist, what is missing, and where the research gaps are. The audit finding, that mainstream models barely document tamper resistance or provenance, is concrete and citable. Staged deployment, watermarking, and provenance are directly usable frameworks for shipping open models. The paper also clarifies that data curation is not the only lever; tamper-resistant training and tampering evaluations are the most underdeveloped parts specific to open weights.
It is a position paper with no experiments, and the 16 problems reflect the authors' judgment of what is under-studied. Scope is limited to technical tools with distinct implications for open weights and explicitly excludes general governance and external guardrail models. The authors acknowledge uncertainty: it is unclear how effective different safeguards will prove, and the counterfactual risk of open versus closed weights is unresolved. They also warn against openness washing and safety washing, treating openness as automatically safe or safety rhetoric as evidence of real safeguards.