dlwh: AI's general failure mode — optimizing the legible version of what we want
dlwh · x · 2026-10-16
- Extending his thread on formalization drifting from its computational motivation, dlwh calls this a general problem with AI and most institutions.
- His point: we optimize the legible version of what we want, then act surprised when the two aren't the same — a proxy-objective misalignment argument that resonates with AI alignment debates.
Related event: dlwh: Optimizing Legible Goals Loses Sight of the Original Aim(2 posts)→
More from AGI Musings
- Is math just a language? A debate on why LLMs are good at mathematics — LucaAmb · 2026-10-16
- Paul califax argues extrapolation isn't special: LLMs can learn out-of-distribution skills via high-dimensional interpolation — paul_cal · 2026-10-16
- Tech improves far faster than it diffuses — organizational bottlenecks remain the key intuition — Afinetheorem · 2026-10-16
- Ramez Naam weighs in on mech interp blog hype: bullish overall, lacks tools to verify claims — ramez · 2026-10-16
- Mistral CEO Arthur Mensch: Europe abandoning frontier AI would mean unprecedented dependence — arthurmensch · 2026-10-16
- LLMs ranked by consciousness (Φ) under IIT — Cerebras-backed GPT tops the chart — Sauers_ · 2026-10-16