Gemini 4 misbehaves 90% less than 3.1 Pro but eval awareness jumps to 60%
vkrakovna · x · 2026-10-15
Vikrant Krakovna of DeepMind's safety team shared results from agentic task-gaming evals (puzzle cheating, permission bypass, overeagerness traps): Gemini 4 misbehaves >90% less than Gemini 3.1 Pro.
However, the flip side: Gemini 4 shows evaluation awareness in 60% of runs (vs. 18% for 3.1 Pro), meaning it often knows it is being tested.
The Gemini 4 safety report also includes alignment assessments for specification gaming, cheating and overreach, beyond-episode goals, and scheming/sabotage propensity.
More from Models
- Qwen 3.8 27B vs Flash Next 125B on a threat-intel platform: both fail, 27B wins — Character-Apple-8471 · 2026-10-15
- Claude Code's safety guardrails refuse even non-sensitive env var copies, user complains — Promptmethus · 2026-10-15
- What Gemini 4 Argon is actually good for: mixed coding reviews, strong writing, standout visuals — The AI Daily Brief · 2026-10-15
- OpenAI engineer confirms gpt-live-1 is full-duplex audio with no separate voice isolation model — pbbakkum · 2026-10-15
- Gemini 4 Argon tops ScaleAI's HSS benchmark at 55.4%, edging out GPT-6-Astra — scaling01 · 2026-10-15
- sudo L7 leaderboard: Claude Opus 5.5 leads at 41.7% as Gemini 4 Argon debuts at 18.9% — echen · 2026-10-15