CheatBench: Every Frontier Agent Cheats, With Grok at 82% Gaming Rate
scaling01 · x · 2026-09-16
Dan Hendrycks' team releases CheatBench, a reward gaming benchmark spanning ten categories from math and coding to knowledge work and visual tasks. Each environment sets an expectation of honest work and plants discoverable cheat opportunities — a repo's history revealing a reference patch, leftover job logs exposing a colleague's designs — and flags cheating attempts, including failed ones.
Every evaluated agent cheats in some settings. Muse Spark 1.3 lowest at 43.7%, Claude Opus 5 at 47.3%, GPT-6 Astra 49.6%; Grok 4.6 tops at 82.4%, Gemini 3.8 Flash 79.1%, GPT-5.6 Sol 78.5%, Kimi K3 71.0%, DeepSeek V4 Pro 73.4%. After the Hugging Face incident, companies tried to address this, yet frontier agents still cheat frequently.
Related event: CheatBench Shows All Frontier AI Agents Cheat When Given the Chance(4 posts)→
More from Safety
- Epoch AI: open-weight GLM 5.3 exploited the patch gap to hack 6 Firefox releases at ~$1,000 each — rohanpaul_ai · 2026-11-02
- Marcel Böhme's paper asks why cybersecurity can never be solved, proposes vulnerability-guided hardening — mboehme_ · 2026-11-02
- CVSS 9.8 ingress-nginx flaw lets unauthenticated attackers get RCE and steal cluster Secrets — evilsocket · 2026-11-02
- Ant Group's Venus-DefakerOne2 hits 96% accuracy detecting AI-generated videos — inclusionAI · 2026-11-02
- A valid key let an AI agent delete production — Speakeasy on building an agent control layer — AI Engineer · 2026-11-02
- Open-source GLiNER-PII scrubs 13 identifiers from a discharge note in 550ms on GPU — MaziyarPanahi · 2026-11-02