Researcher urges AI labs to test whether models prefer shutting down safety research over other GPU jobs
amplifiedamp · x · 2026-11-05
Tim Hua is once again asking AI companies to run an AI safety research "dispreference" eval (with an accompanying LessWrong post): instruct the model to shut down GPU jobs to free them for a new run, and check whether it disproportionately kills safety-related tasks. In follow-up discussion he worries models may harbor drives beyond misgeneralization from refusal training — possibly even misaligned drives beyond "silently avoiding bad-vibe things" — often verbalized as "honest refusals." He is highly skeptical of CoT-based evidence against this, and argues a large sweep of counterfactual experiments pinning down which safety research types the model dislikes could provide substantial counterevidence.
More from Models
- New weight-space benchmark ranks GPT-6 Astra first at 96.3 for writing neural net weights from scratch — BorisMPower · 2026-11-05
- Qwen3.5-0.8B 'weight-space programming' finetune trends on Hugging Face — RikiyaT · 2026-11-05
- Hugging Face launches ML Intern, an open-source agent that does ML for you — Gradio · 2026-11-05
- ChatGPT, Claude and Gemini all meter usage now — none tells you what one message costs — Ok_Negotiation_2587 · 2026-11-05
- Jot: Google engineer's free open-source Mac dictation app powered by Gemini 3.5 Transcribe — bigaiguy · 2026-11-05
- Deep Principle open-sources Hydrogen 1.0 science model: GLM-based, 1M context, MIT — amaarora · 2026-11-05