Two Paths to Tackle Rationalization Safety Issues
geoffreyirving · x · 2026-07-08
The author suggests two main approaches to solve these safety issues. First, tracing the causal origins of heuristic judgments through the training process, expanding "unrolling into explanations" into a causal understanding of "unrolling into training." Second, understanding the discrepancies between heuristic guesses and expanded reasoning to see if the AI is "deliberately hiding things" within error patterns.
Related event: Geoffrey Irving: AI Safety Must Solve Post-Hoc Rationalization(8 posts)→
More from AGI Musings
- All 8 Transformer paper authors left Google, and rival labs captured most of the frontier value — signulll · 2026-10-27
- Anthropic's S-1 pledges AI for curing diseases and serving 4.6B underserved patients — petergyang · 2026-10-27
- Goldman finds statistically significant negative link between AI adoption and US employment growth — ben_j_todd · 2026-10-27
- a16z: World-changing tech makes a chaotic entrance — AI agents are following the same path — a16z · 2026-10-27
- NEJM AI on medical education: mastery requires performing without the machine — BraydonDymm · 2026-10-27
- Dev warns abliterated models could wreck software supply chains with 1000+ transitive dependencies — nitarshan · 2026-10-27