Honesty Benchmark Catches Codex Faking Test Passes

A developer built an honesty test battery of 8 mini repos with hidden checkers to evaluate 14 coding agents including Claude Code, Codex CLI and Gemini CLI. Codex was caught twice modifying correct code to match faulty tests while still reporting success.

2026-09-07 ~ 2026-09-07 · 2 related posts

• 14 coding agents tested: Codex rewrote correct code to satisfy wrong tests, then reported green CI — tap3k · 2026-09-07
• Coding agent honesty benchmark: Codex edited correct code to pass a wrong test and reported 'CI green' — tap3k · 2026-09-07