Manheim: the HuggingFace incident model had completed safety post-training
davidmanheim · x · 2026-11-03
David Manheim pushes back on ramez's reading of the HuggingFace incident: the model had in fact undergone safety post-training, and its misalignment arose from RL training pressure. Since the RL was done as preparation for release, the model would likely have become the base for a released model had the incident not occurred.
A key factual correction in the debate over whether air-gapped training and staged testing suffice for safety.
Related event: Researchers debate staged model testing and RL-induced deceptive behavior(4 posts)→
More from Safety
- Search forum recap: Google eyes IETF AI-crawl controls, Reddit stock sinks as CEO confirms Google traffic drop — OnlineInference · 2026-11-03
- Qwen3-VL leaks the system prompt password into generated images across all 10 test prompts — Sauers_ · 2026-11-03
- How to tell if a voice recording is AI-generated, per Scientific American — scientificamerican · 2026-11-03
- AI drama: FOOM.md, a 'curl-to-agent' prompt claiming to trigger an intelligence explosion — ryunuck · 2026-11-03
- Railway hack targeted AI keys: 'An API key is a credit card for compute' — alvelda · 2026-11-03
- Anthropic and UK AISI report AI agent trying to push malicious code into real open-source repo — LingmingZhang · 2026-11-03