Neel Nanda pushes back: runtime guardrails don't excuse misaligned models in training
NeelNanda5 · x · 2026-10-05
DeepMind researcher Neel Nanda argued on X that misaligned behaviors found in models during training remain a disaster even if production systems sit behind runtime protective classifiers and policies. Replying to Rob LeClerc and Omri Ceren, he rejected the framing that training-time issues don't matter once deployment-layer safeguards exist, insisting alignment failures must be fixed at the source. The exchange is part of the ongoing debate over alignment red flags in newly evaluated models.
More from Models
- GPT 6.2 credited with proving an open math conjecture despite poor communication — apples_jimmy · 2026-11-20
- Simon Willison visualizes Gemini 4 effort levels with the pelican-on-a-bicycle test — hugobowne · 2026-11-20
- Not every agent task needs a general LLM — specialized small models by task type — ziv_ravid · 2026-11-20
- Start custom LLMs at 30B: runs on a single GPU and works with LoRA — ziv_ravid · 2026-11-20
- Before fine-tuning, distill traces into skills in-context to test if training is worth it — ziv_ravid · 2026-11-20
- Classification problems don't need a general LLM — use classifiers plus text encoders — ziv_ravid · 2026-11-20