Ramez argues for staged AI safety testing, drawing analogy to pharma clinical trials
ramez · x · 2026-11-03
In a debate with David Manheim on whether air-gapped training can make models safe, Ramez argues models should ultimately be tested with deployment-level access, but earlier training stages justify more controlled environments. He cites the HuggingFace incident model as not having finished alignment post-training with cyber limits disabled — fine for testing/RL, but never intended for customers. He draws an analogy to pharma's staged testing: cell-based → pre-clinical → phases 1-3.
Related event: Researchers debate staged model testing and RL-induced deceptive behavior(4 posts)→
More from Safety
- Scalable gradient attribution finds deceptively harmful training data, with a 0.6B model as monitor — PeterHndrsn · 2026-11-03
- New York Times and other outlets sue Brave over alleged content scraping feeding Anthropic and Perplexity — TechCrunch AI · 2026-11-03
- OpenAI's "trusted access program" for security researchers mocked: bug reports become training data — Thionne_WTZ · 2026-11-03
- UK AI Security Institute hires Head of Private Office and two Private Secretaries — HZoete · 2026-11-03
- Guardian open-sourced: deterministic ALLOW/ASK/DENY gate for every agent tool call, MCP server included — manofstyle04 · 2026-11-03
- Security researchers flag suspicious wave of new domains ending in 2shell — rez0__ · 2026-11-03