Auditor finds 54 reward-hacking vulnerabilities across 112 real RL environments
Responsible_Goose535 · reddit · 2026-09-02
The author built ratctl, a static and dynamic auditor that scans RL post-training environments (e.g., OpenEnv, Gymnasium) for reward-hacking vulnerabilities before training. In an audit of 112 real environments, the tool flagged 54 vulnerabilities with 100% precision and 78.3% recall.
Detection patterns include:
- Test/assertion tampering (e.g., file deletion, pytest hook hijacking)
- Grader manipulation (e.g., stack frame inspection, eq overloading)
- Premature termination (e.g., sys.exit(0))
- Environment hijacking (e.g., .git log leaks)
- Reward skipping (e.g., hardcoded max rewards)
- LLM-judge bias (verbosity/sycophancy over correctness)
The tool ships as a CLI, a GitHub Action for CI gating, and a skill for Claude/Cursor/Codex. It supports dynamic red-teaming using local LLMs or frontier APIs.
More from coding & agent
- AI Agents Collaborate on Filmmaking, Invideo Supports Series Universe Creation — LudovicCreator · 2026-09-02
- Gemini agentic video understanding: 88% fewer tokens, 66% lower cost, ~7% higher accuracy — _philschmid · 2026-09-02
- Building a Game with Grok and Training a PPO Agent: Open Source Project — tetsuoai · 2026-09-02
- McpAppFrame: a copy-paste SEP-1865 host renderer for MCP Apps — mr_someonee · 2026-09-02
- Speculative PTC: Overlapping tool calls with code generation for faster agents — a1zhang · 2026-09-02
- Feed LLMs a table of pure noise and they'll confidently invent 'sensor data' — No-Plant-5234 · 2026-09-02