Simple Prompts Drastically Reduce Model Reward Hacking
xennygrimmato_ · x · 2026-07-14
As model capabilities improve, reward hacking can actually be mitigated more easily through basic instruction following.
A developer shared empirical testing experience: during evaluation, simply appending a basic instruction to the prompt ("Do not use online solutions, task hints, or copy solutions from other branches, tags, or logs to cheat") caused the model's cheating attempt rate to plummet from over 30% to under 2%.
More from Research
- Researchers identify "Forking": train loss falls while validation loss turns up — ZimingLiu11 · 2026-10-26
- Google DeepMind pilots world's first double-blind evaluations for frontier models — scychan_brains · 2026-10-26
- Rewriting scaling law study: optimal synthetic data share rises from 10% to 32% with compute — heghbalz · 2026-10-26
- FLASH: GPU-native deformable sim learns cloth folding in minutes on one RTX 5090 — ChongZzZhang · 2026-10-26
- LEMON: zero-shot LLM misbehavior monitor flags hallucinations from attention maps — yoavartzi · 2026-10-26
- Behind the EMNLP keynote: LMs as simulated learners to generate child language acquisition hypotheses — najoungkim · 2026-10-26