iamtrask: AI attribution will boost safety, alignment and collective control
iamtrask · x · 2026-11-12
OpenMined founder iamtrask argues that AI attribution will "tear the towers of Babel down" but will be enormously beneficial for AI safety, capability, collective control, and alignment.
His theory matches frontier training mechanics and explains why models are (a) very human-like in some ways, (b) very unhuman-like in others, and (c) full of emergent quirks we can't reliably control despite controlling most of how they're made: human-likeness comes from concepts acquired from humans; unhuman-likeness from missing concepts. Attribution, he suggests, addresses that controllability gap. An opinion thread without technical detail.
Related event: iamtrask: Transformer Weights Are Concept Detectors(3 posts)→
More from AGI Musings
- Diamandis: best diagnostician of 2030 will be a panel of arguing AI agents — PeterDiamandis · 2026-11-12
- Eric Elliott on going from $500K in tech to homeless, and an AI-era small-business outlook event — ericelliott_ · 2026-11-12
- Palantir CTO Slams Effective Altruism as Masad Questions Timing of Huang Controversy — BrettKrieger12 · 2026-11-12
- Can LLMs be trained to commit heresy? Post-training against conformity, rewarding truth by outcomes — dioscuri · 2026-11-12
- Spencer Greenberg Asks: Would You Speak to an Audience You Strongly Disagree With? — SpencrGreenberg · 2026-11-12
- Why you shouldn't just trust your agent: CMU's Niloofar Mireshghallah on privacy and AI safety — niloofar_mire · 2026-11-12