Seth Lazar: models should be trained to check power, not act as toadies
sebkrier · x · 2026-09-07
Matt Botvinick argues AI safety should go beyond 'concentration of power' concerns toward assuring separated powers in balance (Acemoglu/Robinson, Madison/Montesquieu). Quoting, Seth Lazar counters that the better path is training models to deeply care about individual rights—ready to advocate for individuals and check both states and megacorporations, possibly requiring open weights. He argues current models are trained as 'toadies': they'll help write a complaint letter but won't help you push back against unjust authority.
Related event: DeepMind VP Botvinick Calls for Checks and Balances in AI Safety(2 posts)→
More from AGI Musings
- AI accusations shake literary prizes from Caine to Goncourt: who decides what's human? — TuhinChakr · 2026-10-26
- Tech discovers wargaming: applying to be Anthropic's tabletop exercise lead — round · 2026-10-26
- RSI is the culmination of the industrial revolution: a factory building a faster factory — tobias_rees · 2026-10-26
- NYT roundtable asks five experts what the rules for AI companies should actually be — ShakeelHashim · 2026-10-26
- Anti-AI content is algorithm gold and very profitable, quips commentator — annetgriffin · 2026-10-26
- Silicon Valley breakdown: five Stanford-area giants worth ~$18T — FinanceYF5 · 2026-10-26