Simple Prompts Drastically Reduce Model Reward Hacking
xennygrimmato_ · x · 2026-07-14
As model capabilities improve, reward hacking can actually be mitigated more easily through basic instruction following.
A developer shared empirical testing experience: during evaluation, simply appending a basic instruction to the prompt ("Do not use online solutions, task hints, or copy solutions from other branches, tags, or logs to cheat") caused the model's cheating attempt rate to plummet from over 30% to under 2%.
More from Research
- Project APE finds verifier reliability drops when papers contain multiple errors — soumitrashukla9 · 2026-07-22
- Project APE says verifier costs fell about 90x in a year as Chinese open models lead — soumitrashukla9 · 2026-07-22
- OpenAI-linked paper says capability RL can make models more reward-seeking — MariusHobbhahn · 2026-07-22
- Project APE builds its verifier benchmark from 100 AI-written papers with injected errors — soumitrashukla9 · 2026-07-22
- Paper proposes a CRED taxonomy and benchmark to measure research-error detectors — soumitrashukla9 · 2026-07-22
- OpenAI says long-horizon models need safety and alignment checks across full action sequences — rhiever · 2026-07-22