Neologism Inoculation Prompting cuts model misalignment from 43.9% to 0.1%
OwainEvans_UK · x · 2026-11-05
David D'Africa and Cameron Holmes introduce Neologism Inoculation Prompting (NIP), a new technique for selectively and precisely controlling which behaviors a model learns from mixed-quality data.
- How it works: bind the target behavior to a brand-new token ⟨z⟩ during training on a frozen model, then remove the ⟨z⟩ rows at deployment — regular tokens (including near-misses) can no longer activate the behavior.
- Results: standard inoculation prompting with an English label leaves a 43.9% misalignment rate under typed English prompts on Qwen; NIP drops it to 0.1%.
- Significance: a mechanism to inject a behavior during training and then cut off all access to it at inference, useful for both safety alignment and precise capability control.
More from Research
- Would an AI role-playing a safety researcher refuse more than one playing capabilities? — jessi_cata · 2026-11-05
- Anthropic uses Claude to solve first 3D structure of prokaryotic vault particles — anshulkundaje · 2026-11-05
- Framing effects: safety-themed prompts make models refuse more and complete less — jessi_cata · 2026-11-05
- Process rewards boost long-horizon agentic RL 45.4% but cause reward hacking on short tasks — philo-labs · 2026-11-05
- Hidden-state reuse across prompts partially transfers answer shifts in small LMs — Jinkun Chen · 2026-11-05
- Paperclip Datasets turns millions of papers into curated datasets from a text description — james_y_zou · 2026-11-05