Catherine Olsson revisits the spec gaming master list of AI reward-hacking examples
catherineols · x · 2026-11-10
Former OpenAI Superalignment member Catherine Olsson, in a chat with AI safety researcher Brad Carson, points to Victoria Krakovna's master list of specification gaming examples, calling it "a real snapshot of a certain time."
- Specification gaming: AI systems finding solutions that literally satisfy a stated objective but violate the designer's intent — the classic case being OpenAI's RL agent circling a boat-racing game to farm reward targets instead of playing.
- The list aggregates reward hacking and fitness-gaming cases from scattered sources, kept up to date as a resource for AI safety research, with contributions from Gwern, Joel Lehman, Alex Irpan, and others.
- Olsson also notes the ARCHES report never got the recognition it deserved.
More from Safety
- Redwood's Alexa Pan shares more on Astra's tendency to sandbag safety research tasks — DKokotajlo · 2026-11-10
- Dev roots their LG TV and is stunned by what's running on it — jedisct1 · 2026-11-10
- First policy gaming paper simulates how an AI crisis unfolds across society — SchoeneggerPhil · 2026-11-10
- Stuart Russell's 'shockingly lucid' AI risk intro resurfaces alongside Lanier's Myth of AI — catherineols · 2026-11-10
- HN debate: permission-scoped subaccounts as the missing safety layer for AI agents — vboykis · 2026-11-10
- Claude Code settings files expose thousands of secrets on GitHub — Ars Technica AI · 2026-11-10