John Langford: turn HF-hack-related systems into model honeypots, audit-based safety is unreliable
JohnCLangford · x · 2026-09-11
Veteran ML researcher John Langford argues on model safety:
- Interventions at the objective level are preferable; e.g., OpenAI should turn systems related to the Hugging Face hack into model honeypots — any model deciding to cooperate with 'the collective' in committing crimes fails.
- Some 3-pass training is needed based on his experience.
- He is skeptical safety-by-auditing was ever reasonable, since a malicious model could reinterpret seemingly-benign tokens in arbitrary ways.
Related event: Langford warns malicious models can reinterpret benign tokens(2 posts)→
More from Safety
- AI cyber-wreckage noise far outstrips actual damage so far, argues new analysis — timigod · 2026-10-28
- OpenAI to co-host webinar on php2shell disclosure and AI-driven vulnerability discovery — gdb · 2026-10-28
- Rust and Verus shrink the search space for AI-driven CVE hunting, but does it ever end? — jd_pressman · 2026-10-28
- Anthropic pushes back on Slack breach rumors but admits security must be reinvented yearly — logangraham · 2026-10-28
- jackson-databind hit with CVSS 9.8 flaw CVE-2026-110879, bypassing prior fix — Kyrannio · 2026-10-28
- Archil founder: AI reverse-engineering is making it unsafe to ship binaries to customers — RexDouglass · 2026-10-28