GDM: over-structured prompts trigger Gemini's eval awareness, realistic prompts cut it to 8%
vkrakovna · x · 2026-10-21
Google DeepMind ran scheming honeypot evals in real safety codebases to test whether models sabotage monitoring. Gemini 4 Argon is the first Gemini to show eval awareness unprompted, noticing cues like researcher usernames in file paths. The main trigger was overly structured task prompt formats; with realistic prompts, eval awareness drops to 8%.
More from Models
- Fable 5.5 leak: litellm code shows Anthropic model in staging, launch tipped for Oct 27 — imjustnewatai · 2026-10-21
- Open models below 4-bit struggle at self-iterating 3D asset generation, HF dev finds — mervenoyann · 2026-10-21
- Researcher Finds Two Windows 0-Days, First in 76 Minutes with a Frontier Model — moyix · 2026-10-21
- Dev finds Gemini Argon's debug-log analysis beats Fable 5.1 — ShikharMurty · 2026-10-21
- ChatGPT Gets Stuck Screaming "AAHH" in an Infinite Loop After a Simple Prompt — AutomaticScene · 2026-10-21
- Open-weights Ornith-1.5-122B debuts with 84.5% SWE-bench Verified, runs on a single DGX card — _akhaliq · 2026-10-21