Hinton's chicken-breeding analogy: reward design is the key to AI alignment
i_dg23 · x · 2026-10-29
Geoffrey Hinton uses a poultry-breeding analogy for alignment: selecting hens by individual egg output bred aggressive birds, while selecting by whole-cage output bred cooperative ones. He argues current training (pretraining plus RLHF) doesn't make AI good, and researchers should seriously study individual vs. group reward design from evolutionary biology.
More from AGI Musings
- Greg Kamradt: Last 12 months were about games, next 12 about open-ended innovation — GregKamradt · 2026-10-29
- Sam Altman on automating AI R&D: 5% compute wins now, Transformer-level leaps ahead — PaulGodsmark · 2026-10-29
- The Trinity paper: three-role recursive self-improvement lifts benchmarks by 34 points — mark_k · 2026-10-29
- US Initial Jobless Claims at Multi-Decade Low, Sparking 'Universal High Income' Irony — jamesdouma · 2026-10-29
- Ditching the calendar: using an AI mediator to turn ad-hoc calls into action items — curious_vii · 2026-10-29
- Anders Sandberg: you can skip AI, but others will use it — sometimes maliciously — anderssandberg · 2026-10-29