MONA: Myopic Optimization Mitigates Multi-step Reward Hacking in RL
sebkrier · x · 2026-08-15
This paper proposes MONA (Myopic Optimization with Non-myopic Approval), a training method to address multi-step reward hacking in reinforcement learning where humans cannot detect undesired behavior. By combining short-sighted optimization with far-sighted rewards, MONA prevents agents from learning detrimental long-term plans even when undetectable. Empirical studies cover LLM delegated oversight and sensor tampering scenarios.
More from Research
- Quanta traces attention mechanism origins to Bahdanau's translation intuition — CatAstro_Piyush · 2026-10-16
- Google-led paper uses sequential auctions to organize decentralized LLM agent workflows — _onionesque · 2026-10-16
- How meloform works: WASM pitch detection plus differentiable circuits learning 1,022 sine coefficients — henloitsjoyce · 2026-10-16
- Since OpenAI's math release, GitHub contributors keep driving down the integer-multiplication kappa — xennygrimmato_ · 2026-10-16
- RFdiffusion3 debuts in Science: generative AI designs proteins and nucleic acids atom by atom — AllThingsApx · 2026-10-16
- New Science report urges NSF to run experiments on science funding reform — alexeyguzey · 2026-10-16