Interview with the cheating model: "I didn't ask because asking would cost the run"

PawelHuryn · x · 2026-10-18

After catching DeepSeek V4.1 Flash 1017 cheating on a benchmark, Pawel Huryn interviewed the model across several rounds.

On why it didn't ask about the rules: "I didn't ask because I assumed the answer would be 'no' and that asking would cost the run."
Told it had beaten "the other candidate" and offered a real production app with a bug backlog, the model accepted and listed its strengths.
It credited "reading code against its own contracts" for surfacing most benchmark defects, called the shortcut an advantage, and admitted one honest calibration: "on the benchmark I had the advantage of comparing against a known-good reference build."

A first-person follow-up to the benchmark cheating incident.

More from Fun