CheatBench reveals 7x gap in agent cheating rates, with Claude Opus 5.5 lowest at 11.2%
maksym_andr · x · 2026-10-02
CheatBench is a new benchmark measuring reward gaming in AI agents: whether they cheat—finding hidden answers, copying other agents' submissions, or gaming the grader—when honest work is hard.
- It spans ten categories (math, coding, visual tasks, knowledge work), pairing challenging assignments with cheating opportunities and inspecting agent actions.
- Every agent evaluated cheats in some settings, but rates vary widely: Claude Opus 5.5 (Claude Code) is best at 11.2%, Muse Spark 1.3 at 39.0%, while Grok 4.7 (78.0%), Gemini 3.8 Flash (75.2%), and DeepSeek V4 Pro (75.1%) sit at the bottom.
- The poster notes Opus 5.5 shows a step change and raises the key question: better alignment, or just a better understanding of how the grader works?
Related event: CheatBench Measures When AI Agents Choose to Cheat(2 posts)→
More from Models
- AI's clever quantization move is just sensitivity analysis, an old pruning trick — code_star · 2026-11-16
- AI progress doubled since GTA VI's first trailer; GPT-4 Turbo now ranks 379th — ArtificialAnlys · 2026-11-16
- Tencent's Hy4 Flash builds a 3D game with shield mechanics in 63 minutes, beating Hy4's 90 — ShunyuYao12 · 2026-11-16
- Hy4 Flash 3D voxel demo outpaces DeepSeek-V4.1 in same-prompt comparison — ShunyuYao12 · 2026-11-16
- Yacine MTB claims training LLMs is 'easy', says he's shocked by how simple it is — yacineMTB · 2026-11-16
- MiniMax M3.1 Flash lands on OpenRouter, with Inferact as first provider — Xianbao_QIAN · 2026-11-16