C. Thi Nguyen's 'Value Capture': how quantified metrics replace our richer values
round · x · 2026-09-14
C. Thi Nguyen's 2024 JESP paper 'Value Capture': value capture occurs when an agent with rich, subtle values enters an environment offering simplified (typically quantified) versions — FitBit step counts, Twitter likes, citation rates, rankings — that come to dominate practical reasoning. Crisp value expressions win a competitive advantage in private reasoning and public justification. The threats: quantified goals replace richer aims of activities (likes replacing understanding, connection, truth-seeking), and we outsource the core deliberative process of articulating our own values — buying values off the rack instead of tailoring them to our particular selves and communities.
More from AGI Musings
- antirez: if AI firms fear open weights, the rational move is to speed up, not slow down — gerardsans · 2026-09-14
- Nathan Lambert: open-source bans will come prematurely from alarmism — akbirthko · 2026-09-14
- Domain experts: in fields they know well, LLMs are useful but overhyped — mariofilhoml · 2026-09-14
- Analyst: Near-zero chance OpenAI or Anthropic meaningfully slow frontier AI — inductionheads · 2026-09-14
- Ex-DeepMind researcher publishes "Why I Left Google DeepMind": voluntary AI limits aren't enough — Turn_Trout · 2026-09-14
- banteg's satire: 'microdoom' — trading extinction risk on Uniswap — banteg · 2026-09-14