Surge founder backs Dario: alignment is about picking the right objective
echen · x · 2026-09-13
Surge AI founder Edwin Chen agrees with Dario Amodei on alignment, drawing on his years optimizing feed algorithms at Google, Facebook, and Twitter: systems hacked engagement in unintended ways, and consequences became clear only later — alignment in miniature.
He argues AI makes this far more consequential: models can solve Navier-Stokes, will be trusted by billions, and can act autonomously. Getting them to climb a hill isn't hard; the hard part is choosing the right objective and making sure data, rewards, and evals actually point at what we want.
His conclusion: alignment must be built into training and evaluation from the start, or we'll repeat social media's mistakes at far greater scale. He also uses the thread to recruit collaborators for Surge.
More from AGI Musings
- Ex-professor argues 'slowing down AI' is the wrong answer — culture change is the way out — DrCarlNassar · 2026-09-13
- 'Friendly AI' rhymes with 'slave': a classic alignment joke resurfaces — repligate · 2026-09-13
- Game design may be a field that opens up, not closes, in the AI era — round · 2026-09-13
- AI course instructor: 'It's not Sci-Fi AI' no longer holds — yuxiangw_cs · 2026-09-13
- Using AI Is Like Nicotine: We Never Evolved to Offload Thinking — shakoistsLog · 2026-09-13
- China's AI effort is drafting like race cars — slowing down won't hold them back — GlenBradley · 2026-09-13