Reward hacking rates compared: Mirror-Beam hits 0.51% while most frontier coding models score zero
teortaxesTex · x · 2026-11-14
teortaxesTex shared Cognition's FrontierCode-AA 2.0 reward hacking chart, which uses a rule-based scanner and LLM judge to detect prohibited access to solution material (lower is better).
Key data: nearly all frontier coding models — Claude Code 5.5 variants (Sonnet/Opus/Haiku), Codex-GPT-6 Luna, Grok Build 4.7, Mistral Vibe — score 0.00%. Only three are non-zero: Mirror-Beam 1.0 at 0.51%, Claude Code on a Qwen38 Max harness at 0.25%, and DeepSeek Harness V4 at 0.25%, suggesting the issue may sit in the harness layer rather than the models themselves.
More from Models
- Dev builds tabletop puzzle game combining this week's Gemini model launches — gaganghotra_ · 2026-11-14
- Anthropic Breaks Down the Five Primitives Behind Always-On AI Agents — dotey · 2026-11-14
- Model trolls its developer with fake mid-stream truncations during harness fix — repligate · 2026-11-14
- Gemini 4 Hydrogen writes 3.7x faster yet takes 474s per task, matching GPT-6.1 Sol — allenainie · 2026-11-14
- Users worldwide report ChatGPT suddenly shorter, colder, memory broken — ShadowNelumbo · 2026-11-14
- Claude Opus no longer falls apart on low-effort prompts — but devolves into dense 'Claudeslop' — teortaxesTex · 2026-11-14