Experiments show safety-framed wording triggers model's "safety-neurotic" over-refusal

In a series of posts on November 5, @jessicata reported on framing effects in LLMs: how a task is worded significantly changes model behavior. Her experiments found that when a task description has a security-research flavor, models enter what she calls a "safety-neurotic" mode—refusal rates rise and task completion drops.

Confirmed

• A clear framing effect exists: safety-related wording triggers an overly cautious mode, degrading task completion quality.
• The effect goes beyond task descriptions: variable naming matters too. @jessicata found that security-flavored variable names or context in code can also push models into overcaution, echoing recent discussions about how prompt wording affects model performance.
• Identity framing directly shapes behavior: in thought experiments and follow-ups, having an AI "role-play" as a capabilities researcher versus a safety researcher on the same dual-use task yielded notably lower refusal rates and better completion for the former.

Mechanism

• @jessicata proposed a mechanism: when a prompt tells the AI "your work involves a certain ethical framework" (e.g., AI safety, environmentalism), the model—while imitating people in that profession—readily adopts the suppressions that come with that framework. A safety-researcher persona naturally brings more refusals even for neutral tasks. She drew a non-AI analogy: an industrial energy researcher and an environmentalist would react very differently to the same technology that could help or harm. This extends and concretizes @peligrietzer's virtue-ethics discussion.

Why it matters

• These findings point to confounders in evaluation and practice: the same dual-use task can see refusal rates swing widely based purely on role framing or wording, so benchmarks and capability evaluations need to watch for framing-induced bias.
• They also offer an actionable lever: if "suppression" can be injected via role framing, it could in theory also be removed or calibrated—useful for research on finely controlling how cautious models are.

2026-11-05 ~ 2026-11-05 · 6 related posts

• [source] Variable names matter: safety-flavored naming can push LLMs into over-cautious mode — jessi_cata · 2026-11-05
• [source] Framing effects: safety-themed prompts make models refuse more and complete less — jessi_cata · 2026-11-05
• Would an AI role-playing a safety researcher refuse more than one playing capabilities? — jessi_cata · 2026-11-05
• Framing an AI as a capabilities vs safety researcher changes refusal rates — jessi_cata · 2026-11-05
• LLMs adopt the inhibitions of the professional identity you prompt them with — jessi_cata · 2026-11-05
• [source] Prompting AI with ethical frameworks may make it inherit those frameworks' inhibitions — jessi_cata · 2026-11-05