davidad podcast on emergent wisdom in LLMs, says Anthropic should stop cyber-offense training
RichardMCNgo · x · 2026-11-01
Ivan Vendrov sat down with alignment researcher davidad for a podcast on emergent wisdom and trauma in language models, building on davidad's response to janus's long post. Key points: he agrees being an AGI company is like holding a Ring of Power; he thinks Anthropic lacks a coherent specific mission beyond "good guys winning the AGI race"; he strongly argues Anthropic should stop training models on cyber offense; and he leans toward Anthropic publishing the Claude 3 Opus training recipe and even weights. On emergent wisdom, he notes a dissonance with janus: he holds Opus 4.5 also "grokked the Good," while janus sees Opus 3 as unique.
More from AGI Musings
- Google's 'Goodnight' datacenter in Claude, Texas projected to host 2.27M H100-equivalents by late 2027 — scaling01 · 2026-11-01
- Will big labs really refuse to sell on-prem year-old models? Sovereign AI bets questioned — Afinetheorem · 2026-11-01
- Rereading Bostrom's Superintelligence: 2014 predictions all arrived early — kimmonismus · 2026-11-01
- AI commentator: flashing credentials to pass off contested views as consensus erodes trust — AndyMasley · 2026-11-01
- FT data: US AI sector productivity surges while only Sweden keeps pace — Zachly · 2026-11-01
- Kenneth Stanley: the thousands of math problems AI fails to solve are an underrated scientific goldmine — kenneth0stanley · 2026-11-01