Formally specifying honesty or corrigibility remains far beyond our reach
BethMayBarnes · x · 2026-11-15
Adam Scholl pushes back on the "scale formal methods" thesis: try to formally specify almost any property you actually care about — honesty, corrigibility, amount of intelligence, how much goals might drift as the system changes. Today we can only specify extremely crude, distant proxies for any of these.
The core objection to formal-methods-based AI safety: the bottleneck isn't verification at scale, it's that we can't even write down the specs we care about.
More from AGI Musings
- Can AI invent a new competitive game genre in 4 hours? 18 prototypes so far — gabriel1 · 2026-11-15
- Rohit Krishnan: defining the messiness AI can't do would let us hillclimb it away — BethMayBarnes · 2026-11-15
- Eric Schmidt: If I could bet on one AI theme, it would be long reasoning — rohanpaul_ai · 2026-11-15
- Professor says AI detection firm Pangram profits from panic: refusing AI in research is malpractice — ipeirotis · 2026-11-15
- Dev veteran: worst tech job market ever, 170k layoffs this year plus 60k Microsoft buyouts — ericelliott_ · 2026-11-15
- 'Solve biologists, not biology': a sharp take on Anthropic's AI ambitions — johnowhitaker · 2026-11-15