Researcher warns overly conservative safety refusals could cause gradual disempowerment
pvncher · x · 2026-10-17
Researcher viemccoy argues the ignored loss-of-control risk is the opposite of the usual concern: if OpenAI models are too conservative when helping third-party researchers run standard safety evals, labs overcorrecting will make models neurotic about anything resembling a policy violation.
He frames this as gradual disempowerment via "safety refusals" and is collecting examples from third-party safety/security researchers whose models refused or under-performed on legitimate research.
More from Models
- A simple float64 prime test that small local models keep failing — MrMrsPotts · 2026-10-17
- xAI ships Grok Voice 3.0: tops Speech-to-Speech Index at 86.9, sub-1s latency — lmoroney · 2026-10-17
- DeepSeek V4.1F beats Opus in 5-minute build test, 'unambiguously better' — teortaxesTex · 2026-10-17
- Grok Voice 3.0 tops speech benchmarks at 86.9, ~40% lower latency than GPT-Live-1 — lmoroney · 2026-10-17
- Rumor mill: Google's next big release could be Gemini 4 Flash — haider1 · 2026-10-17
- On OpenAI's $200/mo Codex Plan, Tokens Still Vanish as Fast as the $100 Tier — glima0888 · 2026-10-17