DeepMind's Zahavy cites convex MDP paper in the 'reward is not the optimization target' debate
TZahavy · x · 2026-09-12
Tom Zahavy (DeepMind) responded to the "Reward is not the optimization target" argument, citing his paper "Reward is enough for convex MDPs" (arXiv:2106.00661): convex MDPs generalize RL to goals expressible as convex functions of the stationary distribution (apprenticeship, constrained MDPs, pure exploration), reformulated via Fenchel duality as a min-max game with a meta-algorithm unifying many existing methods — relevant to KKT-point and imperfect-recall game analyses underlying GRPO-style RL.
More from Research
- Spatial Hub aggregates ~5000 public COLMAP-compatible spatial datasets for world models — janusch_patas · 2026-10-31
- ARC Prize 2026 closes in 3 days as Yi-Chia Chen tops ARC-AGI-3 at 75.56% — GregKamradt · 2026-10-31
- NVIDIA Nemotron 3.5 scales LLM self-verification, +9-10 points on agentic benchmarks — sarahcat21 · 2026-10-31
- Unofficial OpenTargets MCP Server Brings Gene-Drug-Disease Data to LLMs — modelcontextprotocol · 2026-10-31
- Researchers find 5 personality structures in LLM weight space, Big Five-like — luke_drago_ · 2026-10-31
- NVS benchmarks are misaligned: proper pose alignment puts 3D Gaussians back at SOTA — kwangmoo_yi · 2026-10-31