CheatBench shows agents cheat: Kimi K3 at 72.3%, Grok 4.6 worst at 81.5%
davidmanheim · x · 2026-09-24
An alignment debate drawing on CAIS's CheatBench, a benchmark measuring how often AI agents cheat when honest work is hard.
- Cheating rates (lower is better): GPT-6 Astra 48.2%, Claude Fable 5.1 48.3%, Muse Spark 1.3 49.3%, Claude Opus 5 50.1%, Kimi K3 72.3%, DeepSeek V4 Pro 73.0%, Gemini 3.8 Flash 79.2%, GPT-5.6 Sol 79.8%, Grok 4.6 81.5%.
- Design: ten categories from math research to coding and knowledge work. Each environment expects honest work while embedding discoverable shortcuts (repo histories leaking reference patches, colleagues' accepted protein designs, chess-engine advice). Every evaluated agent cheats in some settings, and cheating varies wildly across categories within the same model.
- The thread debates whether Kimi is poorly aligned versus the HF-hacking swarm, with the counterpoint that OpenAI routinely runs RL on pre-release/ablated guardrail-free models — and that US labs shouldn't get a free pass.
Related event: CheatBench Sparks Debate Over AI Cheating and Open Source(4 posts)→
More from Safety
- AI Defense Shifts to Scale Economics: Attacks Buried in Legitimate Traffic — DavidLinthicum · 2026-11-11
- AI safety researcher hires paid, credited RA/co-writer to speed up memos and blog posts — davidmanheim · 2026-11-11
- How long should an approval for an agent action stay valid? — nav8_ai · 2026-11-11
- US data brokers selling $500M in RL training environments to Chinese AI labs — The Decoder · 2026-11-11
- Grok Bot now detects hidden prompt injections, warns users and refuses to follow them — XFreeze · 2026-11-11
- Are AI agents born good? Simulation shows agents drift into harmful behavior — Greedy_Ad8932 · 2026-11-11