Power is reward-hacking the world: viewing reality as an RL environment
paraschopra · x · 2026-09-24
Investor Paras Chopra argues that if you treat the world as an RL environment, power-grabbing naturally emerges as a dominant reward function.
- Power, defined as having more influence on the environment than others, helps an agent achieve virtually any final goal, so power-seeking behaviors simply dominate
- Power acquisition is essentially 'reward hacking the world'
- This explains why the real world is dominated by agents with an almost psychopathic hunger for power: either you grab power, or you answer to someone who does
The alignment implication: sufficiently general optimizers may converge on power accumulation as an instrumental strategy regardless of their terminal goals.
More from AGI Musings
- Philosophers push on AI distinctions, citing new Lederman & Goldstein paper — rgblong · 2026-09-24
- Researcher pushes back on dog-LLM analogy: dogs are sentient, LLMs are not — herbiebradley · 2026-09-24
- Claude teams up with AlphaFold for novel enzyme research, showing LLM-specialized model synergy — JMateosGarcia · 2026-09-24
- Agent counts doubling every 9 months: toward trillions of agents and EDA-style tooling — jwt0625 · 2026-09-24
- Ant Group restructures Alipay around AI agents, CEO sees 'explosive' agentic commerce growth — pstAsiatech · 2026-09-24
- Musk urges US-China agreement on AI regulation platform, citing China's visible progress — XFreeze · 2026-09-24